2023-03-05 19:53:32,956 INFO [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.mob.compactions.TestMobCompactor timeout: 13 mins 2023-03-05 19:53:33,224 DEBUG [main] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5db36781-e654-c932-96c1-0485fe524a4f 2023-03-05 19:53:33,241 INFO [Time-limited test] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=1, rsPorts=, rsClass=null, numDataNodes=1, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-03-05 19:53:33,242 INFO [Time-limited test] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5db36781-e654-c932-96c1-0485fe524a4f/cluster_01318ba7-3e0b-e9a9-4c5c-f222cae89dc3, deleteOnExit=true 2023-03-05 19:53:33,242 INFO [Time-limited test] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-03-05 19:53:33,243 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5db36781-e654-c932-96c1-0485fe524a4f/test.cache.data in system properties and HBase conf 2023-03-05 19:53:33,243 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5db36781-e654-c932-96c1-0485fe524a4f/hadoop.tmp.dir in system properties and HBase conf 2023-03-05 19:53:33,244 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5db36781-e654-c932-96c1-0485fe524a4f/hadoop.log.dir in system properties and HBase conf 2023-03-05 19:53:33,245 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5db36781-e654-c932-96c1-0485fe524a4f/mapreduce.cluster.local.dir in system properties and HBase conf 2023-03-05 19:53:33,245 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5db36781-e654-c932-96c1-0485fe524a4f/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-03-05 19:53:33,245 INFO [Time-limited test] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-03-05 19:53:33,364 WARN [Time-limited test] util.NativeCodeLoader(62): Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2023-03-05 19:53:33,779 DEBUG [Time-limited test] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-03-05 19:53:33,784 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5db36781-e654-c932-96c1-0485fe524a4f/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-03-05 19:53:33,784 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5db36781-e654-c932-96c1-0485fe524a4f/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-03-05 19:53:33,784 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5db36781-e654-c932-96c1-0485fe524a4f/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-03-05 19:53:33,785 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5db36781-e654-c932-96c1-0485fe524a4f/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-03-05 19:53:33,785 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5db36781-e654-c932-96c1-0485fe524a4f/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-03-05 19:53:33,785 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5db36781-e654-c932-96c1-0485fe524a4f/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-03-05 19:53:33,785 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5db36781-e654-c932-96c1-0485fe524a4f/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-03-05 19:53:33,786 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5db36781-e654-c932-96c1-0485fe524a4f/dfs.journalnode.edits.dir in system properties and HBase conf 2023-03-05 19:53:33,786 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5db36781-e654-c932-96c1-0485fe524a4f/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-03-05 19:53:33,787 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5db36781-e654-c932-96c1-0485fe524a4f/nfs.dump.dir in system properties and HBase conf 2023-03-05 19:53:33,787 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5db36781-e654-c932-96c1-0485fe524a4f/java.io.tmpdir in system properties and HBase conf 2023-03-05 19:53:33,788 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5db36781-e654-c932-96c1-0485fe524a4f/dfs.journalnode.edits.dir in system properties and HBase conf 2023-03-05 19:53:33,788 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5db36781-e654-c932-96c1-0485fe524a4f/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-03-05 19:53:33,788 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5db36781-e654-c932-96c1-0485fe524a4f/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-03-05 19:53:34,383 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-03-05 19:53:34,388 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-03-05 19:53:34,669 WARN [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties 2023-03-05 19:53:34,847 INFO [Time-limited test] log.Slf4jLog(67): Logging to org.slf4j.impl.Reload4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2023-03-05 19:53:34,865 WARN [Time-limited test] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-03-05 19:53:34,902 INFO [Time-limited test] log.Slf4jLog(67): jetty-6.1.26 2023-03-05 19:53:34,946 INFO [Time-limited test] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5db36781-e654-c932-96c1-0485fe524a4f/java.io.tmpdir/Jetty_localhost_36833_hdfs____960933/webapp 2023-03-05 19:53:35,088 INFO [Time-limited test] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:36833 2023-03-05 19:53:35,107 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-03-05 19:53:35,107 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-03-05 19:53:35,661 WARN [Listener at localhost/32809] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-03-05 19:53:35,753 WARN [Listener at localhost/32809] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-03-05 19:53:35,777 WARN [Listener at localhost/32809] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-03-05 19:53:35,785 INFO [Listener at localhost/32809] log.Slf4jLog(67): jetty-6.1.26 2023-03-05 19:53:35,790 INFO [Listener at localhost/32809] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5db36781-e654-c932-96c1-0485fe524a4f/java.io.tmpdir/Jetty_localhost_35991_datanode____lsr9au/webapp 2023-03-05 19:53:35,897 INFO [Listener at localhost/32809] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:35991 2023-03-05 19:53:36,231 WARN [Listener at localhost/40787] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-03-05 19:53:36,759 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xdf8a661536acb9ba: Processing first storage report for DS-ec416333-b957-4f69-9259-854853c39baa from datanode d35f9466-99e1-410c-8caf-7fef1a39b184 2023-03-05 19:53:36,760 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xdf8a661536acb9ba: from storage DS-ec416333-b957-4f69-9259-854853c39baa node DatanodeRegistration(127.0.0.1:41853, datanodeUuid=d35f9466-99e1-410c-8caf-7fef1a39b184, infoPort=34401, infoSecurePort=0, ipcPort=40787, storageInfo=lv=-57;cid=testClusterID;nsid=1941894180;c=1678046014457), blocks: 0, hasStaleStorage: true, processing time: 2 msecs, invalidatedBlocks: 0 2023-03-05 19:53:36,760 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xdf8a661536acb9ba: Processing first storage report for DS-a61562cd-de33-4b75-8599-f8b32efea9e3 from datanode d35f9466-99e1-410c-8caf-7fef1a39b184 2023-03-05 19:53:36,760 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xdf8a661536acb9ba: from storage DS-a61562cd-de33-4b75-8599-f8b32efea9e3 node DatanodeRegistration(127.0.0.1:41853, datanodeUuid=d35f9466-99e1-410c-8caf-7fef1a39b184, infoPort=34401, infoSecurePort=0, ipcPort=40787, storageInfo=lv=-57;cid=testClusterID;nsid=1941894180;c=1678046014457), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-03-05 19:53:36,803 DEBUG [Listener at localhost/40787] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5db36781-e654-c932-96c1-0485fe524a4f 2023-03-05 19:53:36,872 INFO [Listener at localhost/40787] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5db36781-e654-c932-96c1-0485fe524a4f/cluster_01318ba7-3e0b-e9a9-4c5c-f222cae89dc3/zookeeper_0, clientPort=62957, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5db36781-e654-c932-96c1-0485fe524a4f/cluster_01318ba7-3e0b-e9a9-4c5c-f222cae89dc3/zookeeper_0/version-2, dataDirSize=424 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5db36781-e654-c932-96c1-0485fe524a4f/cluster_01318ba7-3e0b-e9a9-4c5c-f222cae89dc3/zookeeper_0/version-2, dataLogSize=424 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, serverId=0 2023-03-05 19:53:36,887 INFO [Listener at localhost/40787] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=62957 2023-03-05 19:53:36,898 INFO [Listener at localhost/40787] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-03-05 19:53:36,901 INFO [Listener at localhost/40787] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-03-05 19:53:37,582 INFO [Listener at localhost/40787] util.FSUtils(471): Created version file at hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743 with version=8 2023-03-05 19:53:37,582 INFO [Listener at localhost/40787] hbase.HBaseTestingUtility(1406): Setting hbase.fs.tmp.dir to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/hbase-staging 2023-03-05 19:53:37,926 INFO [Listener at localhost/40787] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl 2023-03-05 19:53:38,410 INFO [Listener at localhost/40787] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=3 2023-03-05 19:53:38,442 INFO [Listener at localhost/40787] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-03-05 19:53:38,442 INFO [Listener at localhost/40787] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-03-05 19:53:38,442 INFO [Listener at localhost/40787] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-03-05 19:53:38,443 INFO [Listener at localhost/40787] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-03-05 19:53:38,443 INFO [Listener at localhost/40787] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-03-05 19:53:38,593 INFO [Listener at localhost/40787] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-03-05 19:53:38,674 DEBUG [Listener at localhost/40787] util.ClassSize(228): Using Unsafe to estimate memory layout 2023-03-05 19:53:38,768 INFO [Listener at localhost/40787] ipc.NettyRpcServer(120): Bind to /172.31.14.131:38221 2023-03-05 19:53:38,778 INFO [Listener at localhost/40787] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-03-05 19:53:38,780 INFO [Listener at localhost/40787] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-03-05 19:53:38,802 INFO [Listener at localhost/40787] zookeeper.RecoverableZooKeeper(93): Process identifier=master:38221 connecting to ZooKeeper ensemble=127.0.0.1:62957 2023-03-05 19:53:38,855 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:382210x0, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-03-05 19:53:38,858 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): master:38221-0x100b63e1f1d0000 connected 2023-03-05 19:53:38,891 DEBUG [Listener at localhost/40787] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-03-05 19:53:38,892 DEBUG [Listener at localhost/40787] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-03-05 19:53:38,895 DEBUG [Listener at localhost/40787] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-03-05 19:53:38,904 DEBUG [Listener at localhost/40787] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=38221 2023-03-05 19:53:38,904 DEBUG [Listener at localhost/40787] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=38221 2023-03-05 19:53:38,905 DEBUG [Listener at localhost/40787] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=38221 2023-03-05 19:53:38,905 DEBUG [Listener at localhost/40787] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=38221 2023-03-05 19:53:38,906 DEBUG [Listener at localhost/40787] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=38221 2023-03-05 19:53:38,912 INFO [Listener at localhost/40787] master.HMaster(444): hbase.rootdir=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743, hbase.cluster.distributed=false 2023-03-05 19:53:38,984 INFO [Listener at localhost/40787] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=3 2023-03-05 19:53:38,984 INFO [Listener at localhost/40787] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-03-05 19:53:38,984 INFO [Listener at localhost/40787] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-03-05 19:53:38,984 INFO [Listener at localhost/40787] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-03-05 19:53:38,984 INFO [Listener at localhost/40787] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-03-05 19:53:38,984 INFO [Listener at localhost/40787] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-03-05 19:53:38,989 INFO [Listener at localhost/40787] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-03-05 19:53:38,992 INFO [Listener at localhost/40787] ipc.NettyRpcServer(120): Bind to /172.31.14.131:37037 2023-03-05 19:53:38,994 INFO [Listener at localhost/40787] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-03-05 19:53:39,000 DEBUG [Listener at localhost/40787] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-03-05 19:53:39,001 INFO [Listener at localhost/40787] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-03-05 19:53:39,004 INFO [Listener at localhost/40787] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-03-05 19:53:39,006 INFO [Listener at localhost/40787] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:37037 connecting to ZooKeeper ensemble=127.0.0.1:62957 2023-03-05 19:53:39,009 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:370370x0, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-03-05 19:53:39,010 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:37037-0x100b63e1f1d0001 connected 2023-03-05 19:53:39,011 DEBUG [Listener at localhost/40787] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-03-05 19:53:39,012 DEBUG [Listener at localhost/40787] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-03-05 19:53:39,013 DEBUG [Listener at localhost/40787] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-03-05 19:53:39,014 DEBUG [Listener at localhost/40787] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=37037 2023-03-05 19:53:39,014 DEBUG [Listener at localhost/40787] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=37037 2023-03-05 19:53:39,014 DEBUG [Listener at localhost/40787] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=37037 2023-03-05 19:53:39,015 DEBUG [Listener at localhost/40787] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=37037 2023-03-05 19:53:39,015 DEBUG [Listener at localhost/40787] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=37037 2023-03-05 19:53:39,018 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /hbase/backup-masters/jenkins-hbase4.apache.org,38221,1678046017738 2023-03-05 19:53:39,029 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-03-05 19:53:39,030 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/jenkins-hbase4.apache.org,38221,1678046017738 2023-03-05 19:53:39,050 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-03-05 19:53:39,050 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-03-05 19:53:39,050 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-05 19:53:39,051 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-03-05 19:53:39,053 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /hbase/backup-masters/jenkins-hbase4.apache.org,38221,1678046017738 from backup master directory 2023-03-05 19:53:39,053 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-03-05 19:53:39,057 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/jenkins-hbase4.apache.org,38221,1678046017738 2023-03-05 19:53:39,057 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-03-05 19:53:39,058 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-03-05 19:53:39,058 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase4.apache.org,38221,1678046017738 2023-03-05 19:53:39,061 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating data MemStoreChunkPool with chunk size 2 MB, max count 352, initial count 0 2023-03-05 19:53:39,062 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating index MemStoreChunkPool with chunk size 204.80 KB, max count 391, initial count 0 2023-03-05 19:53:39,553 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] util.FSUtils(620): Created cluster ID file at hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/hbase.id with ID: 61900a27-486d-4720-bce7-d8d458c111c1 2023-03-05 19:53:39,595 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-03-05 19:53:39,612 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-05 19:53:40,055 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x04dc6a2d to 127.0.0.1:62957 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-05 19:53:40,086 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5fce74e6, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-05 19:53:40,110 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-03-05 19:53:40,111 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-03-05 19:53:40,128 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(264): ClientProtocol::create wrong number of arguments, should be hadoop 3.2 or below 2023-03-05 19:53:40,129 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(270): ClientProtocol::create wrong number of arguments, should be hadoop 2.x 2023-03-05 19:53:40,130 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(279): can not find SHOULD_REPLICATE flag, should be hadoop 2.x java.lang.IllegalArgumentException: No enum constant org.apache.hadoop.fs.CreateFlag.SHOULD_REPLICATE at java.lang.Enum.valueOf(Enum.java:238) at org.apache.hadoop.fs.CreateFlag.valueOf(CreateFlag.java:63) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.loadShouldReplicateFlag(FanOutOneBlockAsyncDFSOutputHelper.java:277) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.(FanOutOneBlockAsyncDFSOutputHelper.java:304) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:139) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:855) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2193) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:528) at java.lang.Thread.run(Thread.java:750) 2023-03-05 19:53:40,134 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(243): No decryptEncryptedDataEncryptionKey method in DFSClient, should be hadoop version with HDFS-12396 java.lang.NoSuchMethodException: org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(org.apache.hadoop.fs.FileEncryptionInfo) at java.lang.Class.getDeclaredMethod(Class.java:2130) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelperWithoutHDFS12396(FanOutOneBlockAsyncDFSOutputSaslHelper.java:182) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelper(FanOutOneBlockAsyncDFSOutputSaslHelper.java:241) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.(FanOutOneBlockAsyncDFSOutputSaslHelper.java:252) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:140) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:855) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2193) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:528) at java.lang.Thread.run(Thread.java:750) 2023-03-05 19:53:40,136 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-03-05 19:53:40,173 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7693): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/MasterData/data/master/store-tmp 2023-03-05 19:53:40,205 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-05 19:53:40,205 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-03-05 19:53:40,205 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-03-05 19:53:40,205 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-03-05 19:53:40,205 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-03-05 19:53:40,205 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-03-05 19:53:40,205 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-03-05 19:53:40,205 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-03-05 19:53:40,207 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/MasterData/WALs/jenkins-hbase4.apache.org,38221,1678046017738 2023-03-05 19:53:40,236 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C38221%2C1678046017738, suffix=, logDir=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/MasterData/WALs/jenkins-hbase4.apache.org,38221,1678046017738, archiveDir=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/MasterData/oldWALs, maxLogs=10 2023-03-05 19:53:40,310 DEBUG [RS-EventLoopGroup-3-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:41853,DS-ec416333-b957-4f69-9259-854853c39baa,DISK] 2023-03-05 19:53:40,321 DEBUG [RS-EventLoopGroup-3-2] asyncfs.ProtobufDecoder(123): Hadoop 3.2 and below use unshaded protobuf. java.lang.ClassNotFoundException: org.apache.hadoop.thirdparty.protobuf.MessageLite at java.net.URLClassLoader.findClass(URLClassLoader.java:387) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.io.asyncfs.ProtobufDecoder.(ProtobufDecoder.java:118) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.processWriteBlockResponse(FanOutOneBlockAsyncDFSOutputHelper.java:340) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$100(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$4.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:424) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:557) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.addListener(DefaultPromise.java:185) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:418) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:476) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:471) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:625) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:105) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:84) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:653) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:691) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-03-05 19:53:40,393 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/MasterData/WALs/jenkins-hbase4.apache.org,38221,1678046017738/jenkins-hbase4.apache.org%2C38221%2C1678046017738.1678046020249 2023-03-05 19:53:40,394 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:41853,DS-ec416333-b957-4f69-9259-854853c39baa,DISK]] 2023-03-05 19:53:40,395 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-03-05 19:53:40,395 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-05 19:53:40,400 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-03-05 19:53:40,401 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-03-05 19:53:40,469 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-03-05 19:53:40,477 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-03-05 19:53:40,504 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-03-05 19:53:40,517 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-05 19:53:40,524 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-03-05 19:53:40,527 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-03-05 19:53:40,545 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-03-05 19:53:40,559 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-05 19:53:40,560 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60980647, jitterRate=-0.0913175493478775}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-05 19:53:40,560 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-03-05 19:53:40,576 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-03-05 19:53:40,600 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-03-05 19:53:40,600 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-03-05 19:53:40,603 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-03-05 19:53:40,605 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-03-05 19:53:40,642 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 36 msec 2023-03-05 19:53:40,642 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(95): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-03-05 19:53:40,676 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-03-05 19:53:40,683 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-03-05 19:53:40,718 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-03-05 19:53:40,723 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-03-05 19:53:40,726 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/balancer 2023-03-05 19:53:40,733 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-03-05 19:53:40,739 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2023-03-05 19:53:40,742 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-05 19:53:40,743 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2023-03-05 19:53:40,744 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2023-03-05 19:53:40,755 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/snapshot-cleanup 2023-03-05 19:53:40,759 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-03-05 19:53:40,759 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-03-05 19:53:40,759 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-05 19:53:40,760 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase4.apache.org,38221,1678046017738, sessionid=0x100b63e1f1d0000, setting cluster-up flag (Was=false) 2023-03-05 19:53:40,777 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-05 19:53:40,782 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/flush-table-proc/acquired, /hbase/flush-table-proc/reached, /hbase/flush-table-proc/abort 2023-03-05 19:53:40,784 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,38221,1678046017738 2023-03-05 19:53:40,789 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-05 19:53:40,795 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/online-snapshot/acquired, /hbase/online-snapshot/reached, /hbase/online-snapshot/abort 2023-03-05 19:53:40,797 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,38221,1678046017738 2023-03-05 19:53:40,800 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/.hbase-snapshot/.tmp 2023-03-05 19:53:40,821 INFO [RS:0;jenkins-hbase4:37037] regionserver.HRegionServer(952): ClusterId : 61900a27-486d-4720-bce7-d8d458c111c1 2023-03-05 19:53:40,825 DEBUG [RS:0;jenkins-hbase4:37037] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-03-05 19:53:40,831 DEBUG [RS:0;jenkins-hbase4:37037] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-03-05 19:53:40,831 DEBUG [RS:0;jenkins-hbase4:37037] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-03-05 19:53:40,836 DEBUG [RS:0;jenkins-hbase4:37037] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-03-05 19:53:40,837 DEBUG [RS:0;jenkins-hbase4:37037] zookeeper.ReadOnlyZKClient(139): Connect 0x5aeef514 to 127.0.0.1:62957 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-05 19:53:40,843 DEBUG [RS:0;jenkins-hbase4:37037] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@54290039, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-05 19:53:40,844 DEBUG [RS:0;jenkins-hbase4:37037] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@50169e64, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-03-05 19:53:40,874 DEBUG [RS:0;jenkins-hbase4:37037] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:37037 2023-03-05 19:53:40,879 INFO [RS:0;jenkins-hbase4:37037] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-03-05 19:53:40,879 INFO [RS:0;jenkins-hbase4:37037] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-03-05 19:53:40,879 DEBUG [RS:0;jenkins-hbase4:37037] regionserver.HRegionServer(1023): About to register with Master. 2023-03-05 19:53:40,882 INFO [RS:0;jenkins-hbase4:37037] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,38221,1678046017738 with isa=jenkins-hbase4.apache.org/172.31.14.131:37037, startcode=1678046018983 2023-03-05 19:53:40,900 DEBUG [RS:0;jenkins-hbase4:37037] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-03-05 19:53:40,940 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-03-05 19:53:40,950 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-03-05 19:53:40,950 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-03-05 19:53:40,951 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-03-05 19:53:40,951 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-03-05 19:53:40,951 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-03-05 19:53:40,951 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-03-05 19:53:40,951 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-03-05 19:53:40,951 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-03-05 19:53:40,953 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1678046050953 2023-03-05 19:53:40,956 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-03-05 19:53:40,959 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-03-05 19:53:40,959 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-03-05 19:53:40,963 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:49599, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=RegionServerStatusService 2023-03-05 19:53:40,964 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-03-05 19:53:40,969 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-03-05 19:53:40,977 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-03-05 19:53:40,977 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-03-05 19:53:40,977 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server is not running yet at org.apache.hadoop.hbase.master.HMaster.checkServiceStarted(HMaster.java:2832) at org.apache.hadoop.hbase.master.MasterRpcServices.regionServerStartup(MasterRpcServices.java:577) at org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:15952) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-03-05 19:53:40,978 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-03-05 19:53:40,978 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-03-05 19:53:40,980 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-03-05 19:53:40,981 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-03-05 19:53:40,983 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-03-05 19:53:40,984 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-03-05 19:53:40,987 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-03-05 19:53:40,987 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-03-05 19:53:40,990 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1678046020989,5,FailOnTimeoutGroup] 2023-03-05 19:53:40,990 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1678046020990,5,FailOnTimeoutGroup] 2023-03-05 19:53:40,990 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-03-05 19:53:40,991 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-03-05 19:53:40,992 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-03-05 19:53:40,993 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-03-05 19:53:41,014 DEBUG [RS:0;jenkins-hbase4:37037] regionserver.HRegionServer(2829): Master is not running yet 2023-03-05 19:53:41,014 WARN [RS:0;jenkins-hbase4:37037] regionserver.HRegionServer(1031): reportForDuty failed; sleeping 100 ms and then retrying. 2023-03-05 19:53:41,115 INFO [RS:0;jenkins-hbase4:37037] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,38221,1678046017738 with isa=jenkins-hbase4.apache.org/172.31.14.131:37037, startcode=1678046018983 2023-03-05 19:53:41,121 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:41,129 DEBUG [RS:0;jenkins-hbase4:37037] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743 2023-03-05 19:53:41,129 DEBUG [RS:0;jenkins-hbase4:37037] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost:32809 2023-03-05 19:53:41,129 DEBUG [RS:0;jenkins-hbase4:37037] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-03-05 19:53:41,135 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-03-05 19:53:41,136 DEBUG [RS:0;jenkins-hbase4:37037] zookeeper.ZKUtil(162): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:41,137 WARN [RS:0;jenkins-hbase4:37037] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-03-05 19:53:41,137 INFO [RS:0;jenkins-hbase4:37037] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-03-05 19:53:41,137 DEBUG [RS:0;jenkins-hbase4:37037] regionserver.HRegionServer(1947): logDir=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/WALs/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:41,138 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,37037,1678046018983] 2023-03-05 19:53:41,147 DEBUG [RS:0;jenkins-hbase4:37037] zookeeper.ZKUtil(162): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:41,157 DEBUG [RS:0;jenkins-hbase4:37037] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-03-05 19:53:41,165 INFO [RS:0;jenkins-hbase4:37037] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-03-05 19:53:41,186 INFO [RS:0;jenkins-hbase4:37037] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-03-05 19:53:41,189 INFO [RS:0;jenkins-hbase4:37037] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-05 19:53:41,189 INFO [RS:0;jenkins-hbase4:37037] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-05 19:53:41,190 INFO [RS:0;jenkins-hbase4:37037] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-03-05 19:53:41,197 INFO [RS:0;jenkins-hbase4:37037] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=100, unit=MILLISECONDS is enabled. 2023-03-05 19:53:41,197 DEBUG [RS:0;jenkins-hbase4:37037] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-03-05 19:53:41,198 DEBUG [RS:0;jenkins-hbase4:37037] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-03-05 19:53:41,198 DEBUG [RS:0;jenkins-hbase4:37037] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-03-05 19:53:41,198 DEBUG [RS:0;jenkins-hbase4:37037] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-03-05 19:53:41,198 DEBUG [RS:0;jenkins-hbase4:37037] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-03-05 19:53:41,198 DEBUG [RS:0;jenkins-hbase4:37037] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-03-05 19:53:41,198 DEBUG [RS:0;jenkins-hbase4:37037] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-03-05 19:53:41,198 DEBUG [RS:0;jenkins-hbase4:37037] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-03-05 19:53:41,198 DEBUG [RS:0;jenkins-hbase4:37037] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-03-05 19:53:41,198 DEBUG [RS:0;jenkins-hbase4:37037] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-03-05 19:53:41,199 INFO [RS:0;jenkins-hbase4:37037] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-03-05 19:53:41,199 INFO [RS:0;jenkins-hbase4:37037] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-03-05 19:53:41,200 INFO [RS:0;jenkins-hbase4:37037] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-03-05 19:53:41,215 INFO [RS:0;jenkins-hbase4:37037] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-03-05 19:53:41,218 INFO [RS:0;jenkins-hbase4:37037] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,37037,1678046018983-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-03-05 19:53:41,238 INFO [RS:0;jenkins-hbase4:37037] regionserver.Replication(203): jenkins-hbase4.apache.org,37037,1678046018983 started 2023-03-05 19:53:41,238 INFO [RS:0;jenkins-hbase4:37037] regionserver.HRegionServer(1638): Serving as jenkins-hbase4.apache.org,37037,1678046018983, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:37037, sessionid=0x100b63e1f1d0001 2023-03-05 19:53:41,238 DEBUG [RS:0;jenkins-hbase4:37037] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-03-05 19:53:41,238 DEBUG [RS:0;jenkins-hbase4:37037] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:41,238 DEBUG [RS:0;jenkins-hbase4:37037] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,37037,1678046018983' 2023-03-05 19:53:41,238 DEBUG [RS:0;jenkins-hbase4:37037] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:53:41,239 DEBUG [RS:0;jenkins-hbase4:37037] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:53:41,239 DEBUG [RS:0;jenkins-hbase4:37037] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-03-05 19:53:41,239 DEBUG [RS:0;jenkins-hbase4:37037] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-03-05 19:53:41,240 DEBUG [RS:0;jenkins-hbase4:37037] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:41,240 DEBUG [RS:0;jenkins-hbase4:37037] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,37037,1678046018983' 2023-03-05 19:53:41,240 DEBUG [RS:0;jenkins-hbase4:37037] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-03-05 19:53:41,240 DEBUG [RS:0;jenkins-hbase4:37037] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-03-05 19:53:41,241 DEBUG [RS:0;jenkins-hbase4:37037] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-03-05 19:53:41,241 INFO [RS:0;jenkins-hbase4:37037] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-03-05 19:53:41,241 INFO [RS:0;jenkins-hbase4:37037] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-03-05 19:53:41,352 INFO [RS:0;jenkins-hbase4:37037] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C37037%2C1678046018983, suffix=, logDir=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/WALs/jenkins-hbase4.apache.org,37037,1678046018983, archiveDir=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/oldWALs, maxLogs=32 2023-03-05 19:53:41,375 DEBUG [RS-EventLoopGroup-3-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:41853,DS-ec416333-b957-4f69-9259-854853c39baa,DISK] 2023-03-05 19:53:41,379 INFO [RS:0;jenkins-hbase4:37037] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/WALs/jenkins-hbase4.apache.org,37037,1678046018983/jenkins-hbase4.apache.org%2C37037%2C1678046018983.1678046021355 2023-03-05 19:53:41,379 DEBUG [RS:0;jenkins-hbase4:37037] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:41853,DS-ec416333-b957-4f69-9259-854853c39baa,DISK]] 2023-03-05 19:53:41,402 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-03-05 19:53:41,403 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-03-05 19:53:41,403 INFO [PEWorker-1] regionserver.HRegion(7675): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743 2023-03-05 19:53:41,817 DEBUG [PEWorker-1] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-05 19:53:41,821 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-03-05 19:53:41,824 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/hbase/meta/1588230740/info 2023-03-05 19:53:41,825 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-03-05 19:53:41,826 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-05 19:53:41,827 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-03-05 19:53:41,830 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/hbase/meta/1588230740/rep_barrier 2023-03-05 19:53:41,830 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-03-05 19:53:41,831 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-05 19:53:41,831 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-03-05 19:53:41,834 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/hbase/meta/1588230740/table 2023-03-05 19:53:41,834 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-03-05 19:53:41,835 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-05 19:53:41,837 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/hbase/meta/1588230740 2023-03-05 19:53:41,838 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/hbase/meta/1588230740 2023-03-05 19:53:41,841 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-03-05 19:53:41,844 DEBUG [PEWorker-1] regionserver.HRegion(1055): writing seq id for 1588230740 2023-03-05 19:53:41,847 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-05 19:53:41,848 INFO [PEWorker-1] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74348811, jitterRate=0.10788361728191376}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-03-05 19:53:41,848 DEBUG [PEWorker-1] regionserver.HRegion(965): Region open journal for 1588230740: 2023-03-05 19:53:41,848 DEBUG [PEWorker-1] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-03-05 19:53:41,848 INFO [PEWorker-1] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-03-05 19:53:41,848 DEBUG [PEWorker-1] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-03-05 19:53:41,848 DEBUG [PEWorker-1] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-03-05 19:53:41,848 DEBUG [PEWorker-1] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-03-05 19:53:41,849 INFO [PEWorker-1] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-03-05 19:53:41,849 DEBUG [PEWorker-1] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-03-05 19:53:41,855 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-03-05 19:53:41,855 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-03-05 19:53:41,862 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-03-05 19:53:41,874 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-03-05 19:53:41,876 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-03-05 19:53:42,028 DEBUG [jenkins-hbase4:38221] assignment.AssignmentManager(2178): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-03-05 19:53:42,032 INFO [PEWorker-2] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,37037,1678046018983, state=OPENING 2023-03-05 19:53:42,040 DEBUG [PEWorker-2] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-03-05 19:53:42,042 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-05 19:53:42,043 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-03-05 19:53:42,045 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,37037,1678046018983}] 2023-03-05 19:53:42,232 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:42,234 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-03-05 19:53:42,238 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:52916, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-03-05 19:53:42,249 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open hbase:meta,,1.1588230740 2023-03-05 19:53:42,249 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-03-05 19:53:42,253 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C37037%2C1678046018983.meta, suffix=.meta, logDir=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/WALs/jenkins-hbase4.apache.org,37037,1678046018983, archiveDir=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/oldWALs, maxLogs=32 2023-03-05 19:53:42,269 DEBUG [RS-EventLoopGroup-3-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:41853,DS-ec416333-b957-4f69-9259-854853c39baa,DISK] 2023-03-05 19:53:42,274 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/WALs/jenkins-hbase4.apache.org,37037,1678046018983/jenkins-hbase4.apache.org%2C37037%2C1678046018983.meta.1678046022254.meta 2023-03-05 19:53:42,274 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:41853,DS-ec416333-b957-4f69-9259-854853c39baa,DISK]] 2023-03-05 19:53:42,275 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-03-05 19:53:42,277 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-03-05 19:53:42,294 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-03-05 19:53:42,299 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-03-05 19:53:42,305 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-03-05 19:53:42,305 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-05 19:53:42,305 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-03-05 19:53:42,305 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-03-05 19:53:42,308 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-03-05 19:53:42,310 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/hbase/meta/1588230740/info 2023-03-05 19:53:42,310 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/hbase/meta/1588230740/info 2023-03-05 19:53:42,310 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-03-05 19:53:42,311 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-05 19:53:42,311 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-03-05 19:53:42,313 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/hbase/meta/1588230740/rep_barrier 2023-03-05 19:53:42,313 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/hbase/meta/1588230740/rep_barrier 2023-03-05 19:53:42,313 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-03-05 19:53:42,314 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-05 19:53:42,314 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-03-05 19:53:42,316 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/hbase/meta/1588230740/table 2023-03-05 19:53:42,316 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/hbase/meta/1588230740/table 2023-03-05 19:53:42,317 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-03-05 19:53:42,317 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-05 19:53:42,319 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/hbase/meta/1588230740 2023-03-05 19:53:42,322 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/hbase/meta/1588230740 2023-03-05 19:53:42,326 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-03-05 19:53:42,328 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-03-05 19:53:42,329 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59014406, jitterRate=-0.12061682343482971}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-03-05 19:53:42,330 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-03-05 19:53:42,339 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1678046022224 2023-03-05 19:53:42,355 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:meta,,1.1588230740 2023-03-05 19:53:42,356 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened hbase:meta,,1.1588230740 2023-03-05 19:53:42,356 INFO [PEWorker-4] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,37037,1678046018983, state=OPEN 2023-03-05 19:53:42,359 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-03-05 19:53:42,359 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-03-05 19:53:42,363 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=3, resume processing ppid=2 2023-03-05 19:53:42,363 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,37037,1678046018983 in 314 msec 2023-03-05 19:53:42,367 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=2, resume processing ppid=1 2023-03-05 19:53:42,368 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 502 msec 2023-03-05 19:53:42,373 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 1.5180 sec 2023-03-05 19:53:42,373 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1678046022373, completionTime=-1 2023-03-05 19:53:42,373 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=1; waited=0ms, expected min=1 server(s), max=1 server(s), master is running 2023-03-05 19:53:42,374 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1519): Joining cluster... 2023-03-05 19:53:42,439 DEBUG [hconnection-0x1608c6c-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-05 19:53:42,443 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:52920, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-05 19:53:42,460 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1531): Number of RegionServers=1 2023-03-05 19:53:42,460 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1678046082460 2023-03-05 19:53:42,460 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1678046142460 2023-03-05 19:53:42,460 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1538): Joined the cluster in 86 msec 2023-03-05 19:53:42,481 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,38221,1678046017738-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-03-05 19:53:42,482 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,38221,1678046017738-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-03-05 19:53:42,482 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,38221,1678046017738-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-03-05 19:53:42,483 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:38221, period=300000, unit=MILLISECONDS is enabled. 2023-03-05 19:53:42,484 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-03-05 19:53:42,489 DEBUG [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(175): 2023-03-05 19:53:42,497 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-03-05 19:53:42,499 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2148): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-03-05 19:53:42,509 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-03-05 19:53:42,511 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-03-05 19:53:42,514 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-05 19:53:42,534 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/.tmp/data/hbase/namespace/25c8546691095e1f6ab386a199ccadff 2023-03-05 19:53:42,537 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/.tmp/data/hbase/namespace/25c8546691095e1f6ab386a199ccadff empty. 2023-03-05 19:53:42,538 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/.tmp/data/hbase/namespace/25c8546691095e1f6ab386a199ccadff 2023-03-05 19:53:42,538 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-03-05 19:53:42,569 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-03-05 19:53:42,572 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7675): creating {ENCODED => 25c8546691095e1f6ab386a199ccadff, NAME => 'hbase:namespace,,1678046022498.25c8546691095e1f6ab386a199ccadff.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/.tmp 2023-03-05 19:53:42,586 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1678046022498.25c8546691095e1f6ab386a199ccadff.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-05 19:53:42,586 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1604): Closing 25c8546691095e1f6ab386a199ccadff, disabling compactions & flushes 2023-03-05 19:53:42,586 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1678046022498.25c8546691095e1f6ab386a199ccadff. 2023-03-05 19:53:42,586 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1678046022498.25c8546691095e1f6ab386a199ccadff. 2023-03-05 19:53:42,586 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1678046022498.25c8546691095e1f6ab386a199ccadff. after waiting 0 ms 2023-03-05 19:53:42,586 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1678046022498.25c8546691095e1f6ab386a199ccadff. 2023-03-05 19:53:42,586 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1838): Closed hbase:namespace,,1678046022498.25c8546691095e1f6ab386a199ccadff. 2023-03-05 19:53:42,586 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1558): Region close journal for 25c8546691095e1f6ab386a199ccadff: 2023-03-05 19:53:42,591 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-03-05 19:53:42,605 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1678046022498.25c8546691095e1f6ab386a199ccadff.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1678046022594"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678046022594"}]},"ts":"1678046022594"} 2023-03-05 19:53:42,631 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-03-05 19:53:42,633 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-05 19:53:42,637 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678046022633"}]},"ts":"1678046022633"} 2023-03-05 19:53:42,641 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-03-05 19:53:42,651 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=25c8546691095e1f6ab386a199ccadff, ASSIGN}] 2023-03-05 19:53:42,653 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=25c8546691095e1f6ab386a199ccadff, ASSIGN 2023-03-05 19:53:42,655 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=25c8546691095e1f6ab386a199ccadff, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,37037,1678046018983; forceNewPlan=false, retain=false 2023-03-05 19:53:42,806 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=25c8546691095e1f6ab386a199ccadff, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:42,807 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1678046022498.25c8546691095e1f6ab386a199ccadff.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1678046022806"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678046022806"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678046022806"}]},"ts":"1678046022806"} 2023-03-05 19:53:42,812 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure 25c8546691095e1f6ab386a199ccadff, server=jenkins-hbase4.apache.org,37037,1678046018983}] 2023-03-05 19:53:42,972 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open hbase:namespace,,1678046022498.25c8546691095e1f6ab386a199ccadff. 2023-03-05 19:53:42,972 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 25c8546691095e1f6ab386a199ccadff, NAME => 'hbase:namespace,,1678046022498.25c8546691095e1f6ab386a199ccadff.', STARTKEY => '', ENDKEY => ''} 2023-03-05 19:53:42,973 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace 25c8546691095e1f6ab386a199ccadff 2023-03-05 19:53:42,974 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1678046022498.25c8546691095e1f6ab386a199ccadff.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-05 19:53:42,974 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 25c8546691095e1f6ab386a199ccadff 2023-03-05 19:53:42,974 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 25c8546691095e1f6ab386a199ccadff 2023-03-05 19:53:42,976 INFO [StoreOpener-25c8546691095e1f6ab386a199ccadff-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 25c8546691095e1f6ab386a199ccadff 2023-03-05 19:53:42,979 DEBUG [StoreOpener-25c8546691095e1f6ab386a199ccadff-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/hbase/namespace/25c8546691095e1f6ab386a199ccadff/info 2023-03-05 19:53:42,979 DEBUG [StoreOpener-25c8546691095e1f6ab386a199ccadff-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/hbase/namespace/25c8546691095e1f6ab386a199ccadff/info 2023-03-05 19:53:42,979 INFO [StoreOpener-25c8546691095e1f6ab386a199ccadff-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 25c8546691095e1f6ab386a199ccadff columnFamilyName info 2023-03-05 19:53:42,980 INFO [StoreOpener-25c8546691095e1f6ab386a199ccadff-1] regionserver.HStore(310): Store=25c8546691095e1f6ab386a199ccadff/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-05 19:53:42,982 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/hbase/namespace/25c8546691095e1f6ab386a199ccadff 2023-03-05 19:53:42,983 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/hbase/namespace/25c8546691095e1f6ab386a199ccadff 2023-03-05 19:53:42,987 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 25c8546691095e1f6ab386a199ccadff 2023-03-05 19:53:42,991 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/hbase/namespace/25c8546691095e1f6ab386a199ccadff/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-05 19:53:42,991 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 25c8546691095e1f6ab386a199ccadff; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65959316, jitterRate=-0.017129600048065186}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-05 19:53:42,991 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 25c8546691095e1f6ab386a199ccadff: 2023-03-05 19:53:42,993 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:namespace,,1678046022498.25c8546691095e1f6ab386a199ccadff., pid=6, masterSystemTime=1678046022966 2023-03-05 19:53:42,997 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:namespace,,1678046022498.25c8546691095e1f6ab386a199ccadff. 2023-03-05 19:53:42,997 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened hbase:namespace,,1678046022498.25c8546691095e1f6ab386a199ccadff. 2023-03-05 19:53:42,999 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=25c8546691095e1f6ab386a199ccadff, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:42,999 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1678046022498.25c8546691095e1f6ab386a199ccadff.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1678046022998"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678046022998"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678046022998"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678046022998"}]},"ts":"1678046022998"} 2023-03-05 19:53:43,007 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=6, resume processing ppid=5 2023-03-05 19:53:43,008 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure 25c8546691095e1f6ab386a199ccadff, server=jenkins-hbase4.apache.org,37037,1678046018983 in 192 msec 2023-03-05 19:53:43,011 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=5, resume processing ppid=4 2023-03-05 19:53:43,011 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=25c8546691095e1f6ab386a199ccadff, ASSIGN in 356 msec 2023-03-05 19:53:43,012 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-05 19:53:43,013 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678046023013"}]},"ts":"1678046023013"} 2023-03-05 19:53:43,016 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-03-05 19:53:43,019 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-03-05 19:53:43,022 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 520 msec 2023-03-05 19:53:43,111 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/namespace 2023-03-05 19:53:43,113 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/namespace 2023-03-05 19:53:43,113 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-05 19:53:43,153 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-03-05 19:53:43,172 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-03-05 19:53:43,181 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 38 msec 2023-03-05 19:53:43,187 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-03-05 19:53:43,202 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-03-05 19:53:43,207 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 19 msec 2023-03-05 19:53:43,214 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-03-05 19:53:43,216 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2023-03-05 19:53:43,216 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 4.158sec 2023-03-05 19:53:43,218 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-03-05 19:53:43,219 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-03-05 19:53:43,220 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-03-05 19:53:43,220 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,38221,1678046017738-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-03-05 19:53:43,222 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,38221,1678046017738-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-03-05 19:53:43,227 DEBUG [Listener at localhost/40787] zookeeper.ReadOnlyZKClient(139): Connect 0x0629018b to 127.0.0.1:62957 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-05 19:53:43,235 DEBUG [Listener at localhost/40787] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@3c1e7f16, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-05 19:53:43,237 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-03-05 19:53:43,247 DEBUG [hconnection-0x77bc4ff1-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-05 19:53:43,258 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:52928, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-05 19:53:43,267 INFO [Listener at localhost/40787] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase4.apache.org,38221,1678046017738 2023-03-05 19:53:43,268 DEBUG [Listener at localhost/40787] zookeeper.ReadOnlyZKClient(139): Connect 0x7defec53 to 127.0.0.1:62957 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-05 19:53:43,273 DEBUG [Listener at localhost/40787] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@64b944ad, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-05 19:53:43,273 INFO [Listener at localhost/40787] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-03-05 19:53:43,294 INFO [Listener at localhost/40787] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthly Thread=211, OpenFileDescriptor=475, MaxFileDescriptor=60000, SystemLoadAverage=361, ProcessCount=171, AvailableMemoryMB=5259 2023-03-05 19:53:43,305 DEBUG [Listener at localhost/40787] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-05 19:53:43,308 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:34398, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-05 19:53:43,321 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithWeeklyFollowedByMonthly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-03-05 19:53:43,323 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure2.ProcedureExecutor(1029): Stored pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:43,324 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_PRE_OPERATION 2023-03-05 19:53:43,327 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-05 19:53:43,329 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithWeeklyFollowedByMonthly" procId is: 9 2023-03-05 19:53:43,330 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17 2023-03-05 19:53:43,331 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17 empty. 2023-03-05 19:53:43,332 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17 2023-03-05 19:53:43,332 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithWeeklyFollowedByMonthly regions 2023-03-05 19:53:43,338 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-03-05 19:53:43,445 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-03-05 19:53:43,647 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-03-05 19:53:43,749 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/.tabledesc/.tableinfo.0000000001 2023-03-05 19:53:43,752 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(7675): creating {ENCODED => 04217a89b4b7147d2fd1e001ac928c17, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthly,,1678046023317.04217a89b4b7147d2fd1e001ac928c17.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithWeeklyFollowedByMonthly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/.tmp 2023-03-05 19:53:43,767 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthly,,1678046023317.04217a89b4b7147d2fd1e001ac928c17.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-05 19:53:43,767 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1604): Closing 04217a89b4b7147d2fd1e001ac928c17, disabling compactions & flushes 2023-03-05 19:53:43,767 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthly,,1678046023317.04217a89b4b7147d2fd1e001ac928c17. 2023-03-05 19:53:43,767 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1678046023317.04217a89b4b7147d2fd1e001ac928c17. 2023-03-05 19:53:43,767 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1678046023317.04217a89b4b7147d2fd1e001ac928c17. after waiting 0 ms 2023-03-05 19:53:43,767 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthly,,1678046023317.04217a89b4b7147d2fd1e001ac928c17. 2023-03-05 19:53:43,768 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthly,,1678046023317.04217a89b4b7147d2fd1e001ac928c17. 2023-03-05 19:53:43,768 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1558): Region close journal for 04217a89b4b7147d2fd1e001ac928c17: 2023-03-05 19:53:43,773 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_ADD_TO_META 2023-03-05 19:53:43,775 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1678046023317.04217a89b4b7147d2fd1e001ac928c17.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1678046023775"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678046023775"}]},"ts":"1678046023775"} 2023-03-05 19:53:43,778 INFO [PEWorker-5] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-03-05 19:53:43,780 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-05 19:53:43,780 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthly","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678046023780"}]},"ts":"1678046023780"} 2023-03-05 19:53:43,783 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthly, state=ENABLING in hbase:meta 2023-03-05 19:53:43,789 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=04217a89b4b7147d2fd1e001ac928c17, ASSIGN}] 2023-03-05 19:53:43,791 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=04217a89b4b7147d2fd1e001ac928c17, ASSIGN 2023-03-05 19:53:43,793 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=04217a89b4b7147d2fd1e001ac928c17, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,37037,1678046018983; forceNewPlan=false, retain=false 2023-03-05 19:53:43,944 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=04217a89b4b7147d2fd1e001ac928c17, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:43,945 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1678046023317.04217a89b4b7147d2fd1e001ac928c17.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1678046023944"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678046023944"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678046023944"}]},"ts":"1678046023944"} 2023-03-05 19:53:43,948 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=11, ppid=10, state=RUNNABLE; OpenRegionProcedure 04217a89b4b7147d2fd1e001ac928c17, server=jenkins-hbase4.apache.org,37037,1678046018983}] 2023-03-05 19:53:43,949 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-03-05 19:53:44,106 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthly,,1678046023317.04217a89b4b7147d2fd1e001ac928c17. 2023-03-05 19:53:44,107 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 04217a89b4b7147d2fd1e001ac928c17, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthly,,1678046023317.04217a89b4b7147d2fd1e001ac928c17.', STARTKEY => '', ENDKEY => ''} 2023-03-05 19:53:44,107 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthly 04217a89b4b7147d2fd1e001ac928c17 2023-03-05 19:53:44,107 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthly,,1678046023317.04217a89b4b7147d2fd1e001ac928c17.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-05 19:53:44,107 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 04217a89b4b7147d2fd1e001ac928c17 2023-03-05 19:53:44,107 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 04217a89b4b7147d2fd1e001ac928c17 2023-03-05 19:53:44,110 INFO [StoreOpener-04217a89b4b7147d2fd1e001ac928c17-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 04217a89b4b7147d2fd1e001ac928c17 2023-03-05 19:53:44,112 DEBUG [StoreOpener-04217a89b4b7147d2fd1e001ac928c17-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/family1 2023-03-05 19:53:44,113 DEBUG [StoreOpener-04217a89b4b7147d2fd1e001ac928c17-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/family1 2023-03-05 19:53:44,116 INFO [StoreOpener-04217a89b4b7147d2fd1e001ac928c17-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 04217a89b4b7147d2fd1e001ac928c17 columnFamilyName family1 2023-03-05 19:53:44,119 INFO [StoreOpener-04217a89b4b7147d2fd1e001ac928c17-1] regionserver.HStore(310): Store=04217a89b4b7147d2fd1e001ac928c17/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-05 19:53:44,121 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17 2023-03-05 19:53:44,122 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17 2023-03-05 19:53:44,128 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 04217a89b4b7147d2fd1e001ac928c17 2023-03-05 19:53:44,133 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-05 19:53:44,134 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 04217a89b4b7147d2fd1e001ac928c17; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67333410, jitterRate=0.003345996141433716}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-05 19:53:44,134 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 04217a89b4b7147d2fd1e001ac928c17: 2023-03-05 19:53:44,135 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthly,,1678046023317.04217a89b4b7147d2fd1e001ac928c17., pid=11, masterSystemTime=1678046024101 2023-03-05 19:53:44,138 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthly,,1678046023317.04217a89b4b7147d2fd1e001ac928c17. 2023-03-05 19:53:44,138 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthly,,1678046023317.04217a89b4b7147d2fd1e001ac928c17. 2023-03-05 19:53:44,139 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=04217a89b4b7147d2fd1e001ac928c17, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:44,140 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1678046023317.04217a89b4b7147d2fd1e001ac928c17.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1678046024139"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678046024139"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678046024139"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678046024139"}]},"ts":"1678046024139"} 2023-03-05 19:53:44,146 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=11, resume processing ppid=10 2023-03-05 19:53:44,146 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=11, ppid=10, state=SUCCESS; OpenRegionProcedure 04217a89b4b7147d2fd1e001ac928c17, server=jenkins-hbase4.apache.org,37037,1678046018983 in 195 msec 2023-03-05 19:53:44,149 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=10, resume processing ppid=9 2023-03-05 19:53:44,150 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=10, ppid=9, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=04217a89b4b7147d2fd1e001ac928c17, ASSIGN in 357 msec 2023-03-05 19:53:44,151 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-05 19:53:44,151 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthly","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678046024151"}]},"ts":"1678046024151"} 2023-03-05 19:53:44,154 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthly, state=ENABLED in hbase:meta 2023-03-05 19:53:44,157 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_POST_OPERATION 2023-03-05 19:53:44,160 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=9, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly in 836 msec 2023-03-05 19:53:44,451 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-03-05 19:53:44,451 INFO [Listener at localhost/40787] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithWeeklyFollowedByMonthly, procId: 9 completed 2023-03-05 19:53:44,473 DEBUG [hconnection-0x611a5737-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-05 19:53:44,476 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:33070, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-05 19:53:44,644 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-05 19:53:44,653 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:44,653 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-05 19:53:44,654 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:53:44,654 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-05 19:53:44,654 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-05 19:53:44,655 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:44,655 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:44,657 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:53:44,657 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:44,657 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:53:44,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:53:44,657 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:44,657 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-05 19:53:44,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:44,658 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:44,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-05 19:53:44,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:44,659 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:44,660 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:44,661 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-05 19:53:44,661 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:53:44,661 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-05 19:53:44,662 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-05 19:53:44,662 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-05 19:53:44,662 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678046023317.04217a89b4b7147d2fd1e001ac928c17. 2023-03-05 19:53:44,663 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1678046023317.04217a89b4b7147d2fd1e001ac928c17. started... 2023-03-05 19:53:44,664 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 04217a89b4b7147d2fd1e001ac928c17 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-03-05 19:53:45,206 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015113022f99052f9f54e4ba7bb9610e5a3d6ab to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015113022f99052f9f54e4ba7bb9610e5a3d6ab 2023-03-05 19:53:45,621 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/.tmp/family1/bdb3cf4515f244fb8f09e1ed853027db 2023-03-05 19:53:45,632 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/.tmp/family1/bdb3cf4515f244fb8f09e1ed853027db as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/family1/bdb3cf4515f244fb8f09e1ed853027db 2023-03-05 19:53:45,641 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/family1/bdb3cf4515f244fb8f09e1ed853027db, entries=1000, sequenceid=5, filesize=175.8 K 2023-03-05 19:53:45,644 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 04217a89b4b7147d2fd1e001ac928c17 in 979ms, sequenceid=5, compaction requested=false 2023-03-05 19:53:45,645 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-05 19:53:45,648 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 04217a89b4b7147d2fd1e001ac928c17: 2023-03-05 19:53:45,648 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678046023317.04217a89b4b7147d2fd1e001ac928c17. 2023-03-05 19:53:45,648 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-05 19:53:45,648 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-05 19:53:45,648 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:45,648 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-05 19:53:45,648 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-05 19:53:45,652 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:45,652 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:45,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:45,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:53:45,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:53:45,653 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:45,653 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-05 19:53:45,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:53:45,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:53:45,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:45,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:45,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:53:45,655 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-05 19:53:45,655 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-05 19:53:45,655 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@500d4214[Count = 0] remaining members to acquire global barrier 2023-03-05 19:53:45,655 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:45,656 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:45,656 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:45,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:45,656 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-05 19:53:45,657 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-05 19:53:45,657 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,37037,1678046018983' in zk 2023-03-05 19:53:45,657 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:45,657 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-05 19:53:45,659 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:45,659 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-05 19:53:45,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:45,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:53:45,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:53:45,659 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:53:45,659 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-05 19:53:45,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:53:45,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:53:45,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:45,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:45,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:53:45,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:45,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:45,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,37037,1678046018983': 2023-03-05 19:53:45,662 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-05 19:53:45,662 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-05 19:53:45,662 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-05 19:53:45,662 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-05 19:53:45,663 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:45,663 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-05 19:53:45,665 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:45,665 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:45,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:45,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:53:45,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:53:45,665 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:45,665 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:53:45,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:45,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:53:45,666 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:53:45,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:53:45,666 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:45,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:45,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:45,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:53:45,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:45,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:45,668 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:45,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:53:45,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:45,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:45,676 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:45,676 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:53:45,676 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:45,676 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:53:45,676 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-05 19:53:45,676 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-05 19:53:45,676 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:53:45,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:53:45,676 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:45,676 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:53:45,676 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:45,677 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:45,677 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:45,677 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:45,677 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:53:45,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:53:45,678 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-05 19:53:45,678 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-05 19:53:45,778 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-05 19:53:45,783 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-05 19:53:45,793 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-05 19:53:45,796 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-03-05 19:53:45,796 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:45,796 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-05 19:53:45,796 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:53:45,797 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-05 19:53:45,797 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-05 19:53:45,797 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:45,797 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:45,799 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:45,799 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:53:45,799 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:53:45,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:53:45,799 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:45,799 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-05 19:53:45,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:45,800 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:45,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-05 19:53:45,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:45,800 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:45,801 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-03-05 19:53:45,801 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:45,801 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-05 19:53:45,801 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:53:45,801 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-05 19:53:45,801 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-05 19:53:45,801 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-05 19:53:45,802 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678046023317.04217a89b4b7147d2fd1e001ac928c17. 2023-03-05 19:53:45,802 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1678046023317.04217a89b4b7147d2fd1e001ac928c17. started... 2023-03-05 19:53:45,802 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 04217a89b4b7147d2fd1e001ac928c17 1/1 column families, dataSize=74 B heapSize=408 B 2023-03-05 19:53:46,227 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511280b2d7d49b36348a2aae676dbd9d7faf6 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201511280b2d7d49b36348a2aae676dbd9d7faf6 2023-03-05 19:53:46,640 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/.tmp/family1/6f4e5c4546e84648bb41fcf6383a052a 2023-03-05 19:53:46,654 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/.tmp/family1/6f4e5c4546e84648bb41fcf6383a052a as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/family1/6f4e5c4546e84648bb41fcf6383a052a 2023-03-05 19:53:46,664 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/family1/6f4e5c4546e84648bb41fcf6383a052a, entries=1, sequenceid=9, filesize=5.0 K 2023-03-05 19:53:46,666 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 04217a89b4b7147d2fd1e001ac928c17 in 864ms, sequenceid=9, compaction requested=false 2023-03-05 19:53:46,666 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 04217a89b4b7147d2fd1e001ac928c17: 2023-03-05 19:53:46,666 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678046023317.04217a89b4b7147d2fd1e001ac928c17. 2023-03-05 19:53:46,666 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-05 19:53:46,667 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-05 19:53:46,667 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:46,667 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-05 19:53:46,667 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-05 19:53:46,673 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:46,673 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:46,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:46,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:53:46,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:53:46,673 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:46,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:53:46,673 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-05 19:53:46,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:53:46,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:46,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:46,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:53:46,675 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-05 19:53:46,675 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@76a19e2b[Count = 0] remaining members to acquire global barrier 2023-03-05 19:53:46,675 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-05 19:53:46,676 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:46,677 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:46,677 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:46,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:46,677 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-05 19:53:46,677 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-05 19:53:46,677 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,37037,1678046018983' in zk 2023-03-05 19:53:46,677 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:46,677 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-05 19:53:46,679 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-05 19:53:46,679 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:46,679 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:53:46,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:46,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:53:46,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:53:46,679 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-05 19:53:46,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:53:46,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:53:46,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:46,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:46,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:53:46,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:46,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:46,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,37037,1678046018983': 2023-03-05 19:53:46,684 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-05 19:53:46,684 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-05 19:53:46,684 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-05 19:53:46,684 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-05 19:53:46,684 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:46,684 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-05 19:53:46,690 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:46,690 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:46,690 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:46,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:46,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:53:46,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:53:46,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:46,690 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:53:46,691 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:53:46,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:53:46,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:53:46,691 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:46,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:46,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:46,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:53:46,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:46,692 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:46,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:46,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:53:46,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:46,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:46,696 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:46,696 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:53:46,696 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:46,696 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:53:46,696 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:53:46,696 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-05 19:53:46,697 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-05 19:53:46,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:53:46,696 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:46,697 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:46,697 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:46,697 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:46,696 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:53:46,697 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-03-05 19:53:46,697 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:46,697 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:53:46,697 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-05 19:53:46,697 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-05 19:53:46,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:53:46,797 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-05 19:53:46,800 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-05 19:53:46,809 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-05 19:53:46,814 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-03-05 19:53:46,814 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:46,814 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-05 19:53:46,815 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:53:46,815 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-05 19:53:46,815 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-05 19:53:46,816 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:46,816 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:46,817 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:53:46,817 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:46,817 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:53:46,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:53:46,818 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:46,818 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-05 19:53:46,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:46,819 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:46,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-05 19:53:46,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:46,819 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:46,819 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-03-05 19:53:46,820 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:46,820 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-05 19:53:46,820 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:53:46,820 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-05 19:53:46,820 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-05 19:53:46,821 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-05 19:53:46,821 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678046023317.04217a89b4b7147d2fd1e001ac928c17. 2023-03-05 19:53:46,821 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1678046023317.04217a89b4b7147d2fd1e001ac928c17. started... 2023-03-05 19:53:46,821 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 04217a89b4b7147d2fd1e001ac928c17 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-05 19:53:46,855 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151201f0e77ddb6a9547aa825b10f096f7a98b to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151201f0e77ddb6a9547aa825b10f096f7a98b 2023-03-05 19:53:47,162 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-03-05 19:53:47,163 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-03-05 19:53:47,274 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/.tmp/family1/5c6bd7fd0716428c8e350edbf75d0ba5 2023-03-05 19:53:47,285 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/.tmp/family1/5c6bd7fd0716428c8e350edbf75d0ba5 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/family1/5c6bd7fd0716428c8e350edbf75d0ba5 2023-03-05 19:53:47,293 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/family1/5c6bd7fd0716428c8e350edbf75d0ba5, entries=1, sequenceid=13, filesize=5.0 K 2023-03-05 19:53:47,294 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 04217a89b4b7147d2fd1e001ac928c17 in 473ms, sequenceid=13, compaction requested=true 2023-03-05 19:53:47,294 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 04217a89b4b7147d2fd1e001ac928c17: 2023-03-05 19:53:47,294 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678046023317.04217a89b4b7147d2fd1e001ac928c17. 2023-03-05 19:53:47,294 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-05 19:53:47,295 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-05 19:53:47,295 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:47,295 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-05 19:53:47,295 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-05 19:53:47,297 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:47,297 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:47,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:47,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:53:47,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:53:47,298 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:47,298 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-05 19:53:47,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:53:47,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:53:47,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:47,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:47,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:53:47,300 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-05 19:53:47,300 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@47f80aea[Count = 0] remaining members to acquire global barrier 2023-03-05 19:53:47,300 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-05 19:53:47,300 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:47,302 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:47,302 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:47,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:47,302 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-05 19:53:47,302 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-05 19:53:47,302 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,37037,1678046018983' in zk 2023-03-05 19:53:47,302 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:47,302 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-05 19:53:47,304 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-05 19:53:47,304 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:47,304 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:53:47,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:47,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:53:47,304 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-05 19:53:47,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:53:47,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:53:47,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:53:47,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:47,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:47,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:53:47,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:47,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:47,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,37037,1678046018983': 2023-03-05 19:53:47,308 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-05 19:53:47,308 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-05 19:53:47,308 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-05 19:53:47,308 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-05 19:53:47,309 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:47,309 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-05 19:53:47,311 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:47,311 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:47,311 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:47,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:47,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:53:47,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:53:47,311 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:53:47,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:47,312 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:53:47,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:53:47,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:53:47,313 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:47,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:47,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:47,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:53:47,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:47,321 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:47,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:47,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:53:47,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:47,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:47,326 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:47,326 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:47,326 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:47,326 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:47,326 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:53:47,326 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:53:47,326 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:53:47,326 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:53:47,326 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-05 19:53:47,327 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-05 19:53:47,326 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:47,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:53:47,327 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:47,329 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:47,331 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:53:47,331 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-05 19:53:47,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:53:47,331 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-05 19:53:47,432 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-05 19:53:47,433 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-05 19:53:47,441 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-05 19:53:47,443 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-03-05 19:53:47,443 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:47,443 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-05 19:53:47,444 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:53:47,444 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-05 19:53:47,444 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-05 19:53:47,445 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:47,445 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:47,446 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:47,446 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:53:47,446 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:53:47,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:53:47,447 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:47,447 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-05 19:53:47,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:47,447 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:47,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-05 19:53:47,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:47,448 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:47,448 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-03-05 19:53:47,448 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:47,448 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-05 19:53:47,448 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:53:47,449 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-05 19:53:47,449 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-05 19:53:47,449 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-05 19:53:47,449 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678046023317.04217a89b4b7147d2fd1e001ac928c17. 2023-03-05 19:53:47,449 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1678046023317.04217a89b4b7147d2fd1e001ac928c17. started... 2023-03-05 19:53:47,449 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 04217a89b4b7147d2fd1e001ac928c17 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-05 19:53:47,839 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-05 19:53:47,876 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120589cd5b0c3d6c4db0958e4fbec7c6a668 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015120589cd5b0c3d6c4db0958e4fbec7c6a668 2023-03-05 19:53:47,919 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-03-05 19:53:47,920 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint Metrics about HBase RegionObservers 2023-03-05 19:53:48,291 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/.tmp/family1/4bd214df671540adb8ab271686620731 2023-03-05 19:53:48,301 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/.tmp/family1/4bd214df671540adb8ab271686620731 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/family1/4bd214df671540adb8ab271686620731 2023-03-05 19:53:48,310 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/family1/4bd214df671540adb8ab271686620731, entries=1, sequenceid=17, filesize=5.0 K 2023-03-05 19:53:48,311 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 04217a89b4b7147d2fd1e001ac928c17 in 862ms, sequenceid=17, compaction requested=true 2023-03-05 19:53:48,311 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 04217a89b4b7147d2fd1e001ac928c17: 2023-03-05 19:53:48,311 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678046023317.04217a89b4b7147d2fd1e001ac928c17. 2023-03-05 19:53:48,311 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-05 19:53:48,312 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-05 19:53:48,312 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:48,312 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-05 19:53:48,312 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-05 19:53:48,315 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:48,315 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:48,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:48,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:53:48,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:53:48,316 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:48,316 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-05 19:53:48,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:53:48,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:53:48,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:48,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:48,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:53:48,318 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-05 19:53:48,318 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@65379d2b[Count = 0] remaining members to acquire global barrier 2023-03-05 19:53:48,318 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-05 19:53:48,318 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:48,319 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:48,319 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:48,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:48,319 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-05 19:53:48,319 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-05 19:53:48,319 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,37037,1678046018983' in zk 2023-03-05 19:53:48,319 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:48,319 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-05 19:53:48,321 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-05 19:53:48,321 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:48,321 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:53:48,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:48,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:53:48,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:53:48,321 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-05 19:53:48,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:53:48,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:53:48,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:48,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:48,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:53:48,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:48,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:48,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,37037,1678046018983': 2023-03-05 19:53:48,325 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-05 19:53:48,325 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-05 19:53:48,325 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-05 19:53:48,325 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-05 19:53:48,325 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:48,325 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-05 19:53:48,327 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:48,327 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:48,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:48,327 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:48,327 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:53:48,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:48,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:53:48,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:53:48,328 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:53:48,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:53:48,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:53:48,328 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:48,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:48,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:48,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:53:48,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:48,330 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:48,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:48,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:53:48,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:48,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:48,340 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:53:48,340 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:53:48,340 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:53:48,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:53:48,340 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:48,340 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:48,340 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:48,340 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:48,340 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:48,340 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:48,340 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:48,341 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:53:48,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:53:48,342 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:53:48,342 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-05 19:53:48,343 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-05 19:53:48,342 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-03-05 19:53:48,343 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-05 19:53:48,344 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-05 19:53:48,444 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-05 19:53:48,445 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-05 19:53:48,453 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-05 19:53:48,455 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-03-05 19:53:48,455 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:48,455 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-05 19:53:48,456 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:53:48,456 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-05 19:53:48,456 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-05 19:53:48,457 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:48,457 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:48,458 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:48,458 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:53:48,458 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:53:48,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:53:48,459 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:48,459 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-05 19:53:48,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:48,459 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:48,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-05 19:53:48,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:48,460 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:48,460 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-03-05 19:53:48,460 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:48,460 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-05 19:53:48,460 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:53:48,461 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-05 19:53:48,461 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-05 19:53:48,461 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-05 19:53:48,461 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678046023317.04217a89b4b7147d2fd1e001ac928c17. 2023-03-05 19:53:48,461 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1678046023317.04217a89b4b7147d2fd1e001ac928c17. started... 2023-03-05 19:53:48,461 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 04217a89b4b7147d2fd1e001ac928c17 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-05 19:53:48,885 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015122852779b402a504bd392803354f70b558d to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015122852779b402a504bd392803354f70b558d 2023-03-05 19:53:48,896 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/.tmp/family1/f8200947e118445cb3e84c45d8c916ef 2023-03-05 19:53:48,904 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/.tmp/family1/f8200947e118445cb3e84c45d8c916ef as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/family1/f8200947e118445cb3e84c45d8c916ef 2023-03-05 19:53:48,912 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/family1/f8200947e118445cb3e84c45d8c916ef, entries=1, sequenceid=21, filesize=5.0 K 2023-03-05 19:53:48,913 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 04217a89b4b7147d2fd1e001ac928c17 in 452ms, sequenceid=21, compaction requested=true 2023-03-05 19:53:48,914 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 04217a89b4b7147d2fd1e001ac928c17: 2023-03-05 19:53:48,914 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678046023317.04217a89b4b7147d2fd1e001ac928c17. 2023-03-05 19:53:48,914 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-05 19:53:48,914 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-05 19:53:48,914 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:48,914 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-05 19:53:48,914 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-05 19:53:48,919 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:48,919 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:48,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:48,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:53:48,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:53:48,919 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:48,919 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-05 19:53:48,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:53:48,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:53:48,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:48,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:48,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:53:48,921 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-05 19:53:48,921 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5f906e7b[Count = 0] remaining members to acquire global barrier 2023-03-05 19:53:48,921 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-05 19:53:48,921 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:48,923 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:48,923 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:48,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:48,923 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-05 19:53:48,923 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-05 19:53:48,923 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,37037,1678046018983' in zk 2023-03-05 19:53:48,923 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:48,923 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-05 19:53:48,926 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-05 19:53:48,926 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:48,927 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:53:48,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:48,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:53:48,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:53:48,927 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-05 19:53:48,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:53:48,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:53:48,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:48,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:48,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:53:48,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:48,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:48,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,37037,1678046018983': 2023-03-05 19:53:48,930 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-05 19:53:48,930 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-05 19:53:48,930 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-05 19:53:48,930 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-05 19:53:48,931 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:48,931 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-05 19:53:48,933 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:48,933 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:48,933 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:48,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:48,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:53:48,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:53:48,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:48,933 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:53:48,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:53:48,933 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:53:48,933 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:48,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:53:48,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:48,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:48,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:53:48,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:48,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:48,935 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:48,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:53:48,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:48,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:48,938 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:48,938 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:48,939 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:48,939 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:48,939 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:48,939 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:53:48,939 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:48,939 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:53:48,939 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:48,939 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:53:48,939 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:53:48,939 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-03-05 19:53:48,939 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-05 19:53:48,940 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-05 19:53:48,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:53:48,940 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-05 19:53:48,940 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:53:48,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:53:48,940 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-05 19:53:49,040 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-05 19:53:49,042 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-05 19:53:49,049 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-05 19:53:49,052 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-03-05 19:53:49,052 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:49,052 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-05 19:53:49,052 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:53:49,052 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-05 19:53:49,052 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-05 19:53:49,053 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:49,053 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:49,054 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:53:49,054 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:49,054 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:53:49,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:53:49,055 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:49,055 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-05 19:53:49,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:49,055 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:49,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-05 19:53:49,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:49,056 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:49,056 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-03-05 19:53:49,056 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:49,056 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-05 19:53:49,056 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:53:49,057 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-05 19:53:49,057 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-05 19:53:49,057 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-05 19:53:49,083 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678046023317.04217a89b4b7147d2fd1e001ac928c17. 2023-03-05 19:53:49,083 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1678046023317.04217a89b4b7147d2fd1e001ac928c17. started... 2023-03-05 19:53:49,083 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 04217a89b4b7147d2fd1e001ac928c17 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-05 19:53:49,108 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015123119dc85eb575c4d5bbb744cebcb76689d to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015123119dc85eb575c4d5bbb744cebcb76689d 2023-03-05 19:53:49,118 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/.tmp/family1/71e1523680814c9c83a78eede5abdedc 2023-03-05 19:53:49,125 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/.tmp/family1/71e1523680814c9c83a78eede5abdedc as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/family1/71e1523680814c9c83a78eede5abdedc 2023-03-05 19:53:49,132 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/family1/71e1523680814c9c83a78eede5abdedc, entries=1, sequenceid=25, filesize=5.0 K 2023-03-05 19:53:49,133 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 04217a89b4b7147d2fd1e001ac928c17 in 50ms, sequenceid=25, compaction requested=true 2023-03-05 19:53:49,133 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 04217a89b4b7147d2fd1e001ac928c17: 2023-03-05 19:53:49,133 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678046023317.04217a89b4b7147d2fd1e001ac928c17. 2023-03-05 19:53:49,134 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-05 19:53:49,134 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-05 19:53:49,134 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:49,134 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-05 19:53:49,134 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-05 19:53:49,137 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:49,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:49,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:53:49,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:53:49,138 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:49,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:53:49,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:53:49,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:49,139 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:49,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:49,140 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-05 19:53:49,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:53:49,140 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-05 19:53:49,141 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@253ae364[Count = 0] remaining members to acquire global barrier 2023-03-05 19:53:49,141 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-05 19:53:49,141 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:49,144 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:49,144 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:49,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:49,144 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-05 19:53:49,144 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-05 19:53:49,144 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,37037,1678046018983' in zk 2023-03-05 19:53:49,144 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:49,144 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-05 19:53:49,146 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-05 19:53:49,146 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:49,146 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:53:49,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:49,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:53:49,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:53:49,146 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-05 19:53:49,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:53:49,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:53:49,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:49,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:49,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:53:49,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:49,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:49,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,37037,1678046018983': 2023-03-05 19:53:49,150 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-05 19:53:49,150 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-05 19:53:49,150 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-05 19:53:49,150 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-05 19:53:49,150 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:49,150 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-05 19:53:49,153 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:49,153 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:49,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:49,153 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:49,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:53:49,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:53:49,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:49,153 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:53:49,154 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:49,154 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:53:49,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:53:49,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:53:49,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:49,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:49,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:53:49,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:49,156 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:49,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:49,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:53:49,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:49,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:49,162 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:49,162 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:49,162 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:49,162 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:49,162 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:53:49,162 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-05 19:53:49,163 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-05 19:53:49,163 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-03-05 19:53:49,163 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:53:49,162 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:53:49,162 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:49,163 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:53:49,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:53:49,164 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-05 19:53:49,164 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-05 19:53:49,164 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:49,164 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:53:49,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:53:49,164 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:49,264 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-05 19:53:49,266 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-05 19:53:49,273 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-05 19:53:49,275 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-03-05 19:53:49,275 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:49,275 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-05 19:53:49,275 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:53:49,275 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-05 19:53:49,275 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-05 19:53:49,276 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:49,276 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:49,277 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:53:49,277 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:49,277 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:53:49,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:53:49,278 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:49,278 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-05 19:53:49,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:49,279 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:49,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-05 19:53:49,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:49,279 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:49,279 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-03-05 19:53:49,279 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:49,280 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-05 19:53:49,280 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:53:49,280 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-05 19:53:49,280 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-05 19:53:49,280 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-05 19:53:49,280 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678046023317.04217a89b4b7147d2fd1e001ac928c17. 2023-03-05 19:53:49,280 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1678046023317.04217a89b4b7147d2fd1e001ac928c17. started... 2023-03-05 19:53:49,281 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 04217a89b4b7147d2fd1e001ac928c17 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-05 19:53:49,314 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160101e004855f93994e5b983bf1dbebd3bcd5 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20160101e004855f93994e5b983bf1dbebd3bcd5 2023-03-05 19:53:49,769 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/.tmp/family1/1e9f737fbd214e2b80fbb6018f5a8195 2023-03-05 19:53:49,778 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/.tmp/family1/1e9f737fbd214e2b80fbb6018f5a8195 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/family1/1e9f737fbd214e2b80fbb6018f5a8195 2023-03-05 19:53:49,788 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/family1/1e9f737fbd214e2b80fbb6018f5a8195, entries=1, sequenceid=29, filesize=5.0 K 2023-03-05 19:53:49,790 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 04217a89b4b7147d2fd1e001ac928c17 in 509ms, sequenceid=29, compaction requested=true 2023-03-05 19:53:49,790 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 04217a89b4b7147d2fd1e001ac928c17: 2023-03-05 19:53:49,790 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678046023317.04217a89b4b7147d2fd1e001ac928c17. 2023-03-05 19:53:49,791 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-05 19:53:49,791 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-05 19:53:49,791 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:49,791 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-05 19:53:49,791 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-05 19:53:49,793 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:49,793 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:49,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:49,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:53:49,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:53:49,794 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:49,794 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-05 19:53:49,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:53:49,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:53:49,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:49,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:49,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:53:49,796 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-05 19:53:49,796 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@22921273[Count = 0] remaining members to acquire global barrier 2023-03-05 19:53:49,796 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-05 19:53:49,797 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:49,798 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:49,798 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:49,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:49,798 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-05 19:53:49,798 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-05 19:53:49,798 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,37037,1678046018983' in zk 2023-03-05 19:53:49,798 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:49,799 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-05 19:53:49,801 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-05 19:53:49,801 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:53:49,801 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-05 19:53:49,801 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:49,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:49,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:53:49,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:53:49,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:53:49,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:53:49,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:49,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:49,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:53:49,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:49,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:49,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,37037,1678046018983': 2023-03-05 19:53:49,805 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-05 19:53:49,805 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-05 19:53:49,805 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-05 19:53:49,805 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-05 19:53:49,805 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:49,805 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-05 19:53:49,806 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:49,806 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:53:49,807 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:49,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:49,807 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:49,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:49,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:53:49,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:53:49,807 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:53:49,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:53:49,808 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:49,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:53:49,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:49,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:49,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:53:49,809 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:49,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:49,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:49,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:53:49,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:49,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:49,815 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:49,815 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:49,816 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:49,816 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:53:49,816 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:53:49,816 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:53:49,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:53:49,816 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:53:49,816 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:49,817 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:49,817 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:49,817 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:49,816 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-05 19:53:49,817 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-05 19:53:49,817 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:53:49,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:53:49,817 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-03-05 19:53:49,817 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-05 19:53:49,818 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-05 19:53:49,918 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-05 19:53:49,920 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-05 19:53:49,928 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-05 19:53:49,932 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-03-05 19:53:49,932 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:49,932 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-05 19:53:49,932 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:53:49,933 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-05 19:53:49,933 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-05 19:53:49,933 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:49,933 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:49,936 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:53:49,936 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:49,936 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:53:49,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:53:49,936 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:49,937 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-05 19:53:49,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:49,937 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:49,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-05 19:53:49,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:49,937 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:49,938 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-03-05 19:53:49,938 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:49,938 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-05 19:53:49,938 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:53:49,938 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-05 19:53:49,939 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-05 19:53:49,939 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-05 19:53:49,939 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678046023317.04217a89b4b7147d2fd1e001ac928c17. 2023-03-05 19:53:49,939 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1678046023317.04217a89b4b7147d2fd1e001ac928c17. started... 2023-03-05 19:53:49,939 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 04217a89b4b7147d2fd1e001ac928c17 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-05 19:53:50,369 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103d216c95bc48b4439ba6c3d0fd1c357ff to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20160103d216c95bc48b4439ba6c3d0fd1c357ff 2023-03-05 19:53:50,379 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/.tmp/family1/5ceb942597e9436fb85b550b949953ae 2023-03-05 19:53:50,388 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/.tmp/family1/5ceb942597e9436fb85b550b949953ae as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/family1/5ceb942597e9436fb85b550b949953ae 2023-03-05 19:53:50,396 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/family1/5ceb942597e9436fb85b550b949953ae, entries=1, sequenceid=33, filesize=5.0 K 2023-03-05 19:53:50,397 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 04217a89b4b7147d2fd1e001ac928c17 in 458ms, sequenceid=33, compaction requested=true 2023-03-05 19:53:50,398 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 04217a89b4b7147d2fd1e001ac928c17: 2023-03-05 19:53:50,398 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678046023317.04217a89b4b7147d2fd1e001ac928c17. 2023-03-05 19:53:50,398 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-05 19:53:50,398 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-05 19:53:50,399 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:50,399 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-05 19:53:50,399 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-05 19:53:50,401 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:50,401 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:50,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:50,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:53:50,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:53:50,401 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:50,401 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-05 19:53:50,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:53:50,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:53:50,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:50,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:50,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:53:50,403 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-05 19:53:50,403 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7b950dcf[Count = 0] remaining members to acquire global barrier 2023-03-05 19:53:50,403 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-05 19:53:50,403 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:50,405 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:50,405 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:50,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:50,405 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-05 19:53:50,405 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-05 19:53:50,405 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,37037,1678046018983' in zk 2023-03-05 19:53:50,405 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:50,405 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-05 19:53:50,407 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:50,407 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-05 19:53:50,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:50,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:53:50,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:53:50,408 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:53:50,408 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-05 19:53:50,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:53:50,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:53:50,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:50,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:50,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:53:50,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:50,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:50,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,37037,1678046018983': 2023-03-05 19:53:50,412 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-05 19:53:50,412 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-05 19:53:50,412 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-05 19:53:50,412 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-05 19:53:50,412 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:50,412 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-05 19:53:50,414 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:50,414 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:50,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:50,414 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:50,414 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:53:50,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:50,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:53:50,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:53:50,415 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:53:50,415 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:50,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:53:50,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:53:50,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:50,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:50,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:53:50,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:50,416 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:50,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:50,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:53:50,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:50,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:50,419 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:50,419 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:53:50,419 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:53:50,419 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:53:50,419 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:53:50,419 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-03-05 19:53:50,419 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-05 19:53:50,420 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-05 19:53:50,419 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:50,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:53:50,420 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:50,420 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:50,420 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:50,420 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:50,420 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:50,421 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-05 19:53:50,421 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:53:50,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:53:50,421 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-05 19:53:50,521 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-05 19:53:50,523 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-05 19:53:50,531 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-05 19:53:50,543 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-03-05 19:53:50,543 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:50,543 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-05 19:53:50,543 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:53:50,543 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-05 19:53:50,544 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-05 19:53:50,548 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:50,548 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:50,549 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:53:50,549 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:50,549 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:53:50,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:53:50,550 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:50,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:50,550 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-05 19:53:50,551 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:50,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-05 19:53:50,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:50,551 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:50,551 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-03-05 19:53:50,552 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:50,553 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-05 19:53:50,553 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:53:50,553 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-05 19:53:50,554 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-05 19:53:50,554 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-05 19:53:50,554 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678046023317.04217a89b4b7147d2fd1e001ac928c17. 2023-03-05 19:53:50,554 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1678046023317.04217a89b4b7147d2fd1e001ac928c17. started... 2023-03-05 19:53:50,554 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 04217a89b4b7147d2fd1e001ac928c17 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-05 19:53:50,985 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201509078140bde90bf64d80887995c38883da3c to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201509078140bde90bf64d80887995c38883da3c 2023-03-05 19:53:51,396 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/.tmp/family1/176651469a5f4bd59e089315da8b846d 2023-03-05 19:53:51,406 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/.tmp/family1/176651469a5f4bd59e089315da8b846d as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/family1/176651469a5f4bd59e089315da8b846d 2023-03-05 19:53:51,414 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/family1/176651469a5f4bd59e089315da8b846d, entries=1, sequenceid=37, filesize=5.0 K 2023-03-05 19:53:51,416 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 04217a89b4b7147d2fd1e001ac928c17 in 862ms, sequenceid=37, compaction requested=true 2023-03-05 19:53:51,416 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 04217a89b4b7147d2fd1e001ac928c17: 2023-03-05 19:53:51,416 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678046023317.04217a89b4b7147d2fd1e001ac928c17. 2023-03-05 19:53:51,416 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-05 19:53:51,417 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-05 19:53:51,417 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:51,417 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-05 19:53:51,417 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-05 19:53:51,420 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:51,420 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:51,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:51,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:53:51,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:53:51,420 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:51,420 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-05 19:53:51,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:53:51,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:53:51,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:51,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:51,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:53:51,422 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-05 19:53:51,422 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@272a4770[Count = 0] remaining members to acquire global barrier 2023-03-05 19:53:51,422 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-05 19:53:51,422 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:51,423 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:51,423 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:51,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:51,423 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-05 19:53:51,423 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-05 19:53:51,423 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,37037,1678046018983' in zk 2023-03-05 19:53:51,424 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:51,424 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-05 19:53:51,425 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-05 19:53:51,425 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:51,425 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:53:51,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:51,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:53:51,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:53:51,425 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-05 19:53:51,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:53:51,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:53:51,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:51,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:51,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:53:51,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:51,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:51,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,37037,1678046018983': 2023-03-05 19:53:51,429 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-05 19:53:51,429 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-05 19:53:51,429 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-05 19:53:51,429 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-05 19:53:51,429 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:51,429 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-05 19:53:51,431 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:51,431 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:51,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:51,431 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:51,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:53:51,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:53:51,431 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:53:51,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:51,431 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:51,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:53:51,431 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:53:51,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:53:51,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:51,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:51,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:53:51,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:51,433 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:51,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:51,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:53:51,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:51,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:51,436 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:51,436 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:53:51,436 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:51,436 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:53:51,436 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:51,436 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-05 19:53:51,437 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-05 19:53:51,436 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:53:51,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:53:51,436 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:53:51,437 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-03-05 19:53:51,436 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:51,437 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:51,437 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:51,437 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:51,437 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-05 19:53:51,437 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-05 19:53:51,437 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:53:51,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:53:51,537 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-05 19:53:51,539 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-05 19:53:51,546 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-05 19:53:51,548 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-03-05 19:53:51,548 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:51,548 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-05 19:53:51,548 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:53:51,549 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-05 19:53:51,549 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-05 19:53:51,550 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:51,550 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:51,551 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:53:51,551 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:51,551 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:53:51,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:53:51,551 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:51,551 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-05 19:53:51,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:51,552 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:51,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-05 19:53:51,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:51,554 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:51,555 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-03-05 19:53:51,555 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:51,555 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-05 19:53:51,555 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:53:51,556 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-05 19:53:51,556 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-05 19:53:51,556 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-05 19:53:51,556 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678046023317.04217a89b4b7147d2fd1e001ac928c17. 2023-03-05 19:53:51,556 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1678046023317.04217a89b4b7147d2fd1e001ac928c17. started... 2023-03-05 19:53:51,557 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 04217a89b4b7147d2fd1e001ac928c17 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-05 19:53:51,976 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151120775a3a7933f0436da2e681e1c2e446a8 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151120775a3a7933f0436da2e681e1c2e446a8 2023-03-05 19:53:52,385 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/.tmp/family1/cb5d9649104f454bbdd20323fccb624b 2023-03-05 19:53:52,392 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/.tmp/family1/cb5d9649104f454bbdd20323fccb624b as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/family1/cb5d9649104f454bbdd20323fccb624b 2023-03-05 19:53:52,399 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/family1/cb5d9649104f454bbdd20323fccb624b, entries=1, sequenceid=41, filesize=5.0 K 2023-03-05 19:53:52,401 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 04217a89b4b7147d2fd1e001ac928c17 in 844ms, sequenceid=41, compaction requested=true 2023-03-05 19:53:52,401 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 04217a89b4b7147d2fd1e001ac928c17: 2023-03-05 19:53:52,401 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678046023317.04217a89b4b7147d2fd1e001ac928c17. 2023-03-05 19:53:52,401 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-05 19:53:52,401 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-05 19:53:52,401 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:52,401 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-05 19:53:52,402 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-05 19:53:52,403 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:52,403 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:52,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:52,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:53:52,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:53:52,404 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:52,404 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-05 19:53:52,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:53:52,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:53:52,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:52,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:52,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:53:52,405 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-05 19:53:52,405 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@55f9b132[Count = 0] remaining members to acquire global barrier 2023-03-05 19:53:52,405 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-05 19:53:52,405 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:52,407 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:52,407 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:52,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:52,407 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-05 19:53:52,407 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-05 19:53:52,407 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,37037,1678046018983' in zk 2023-03-05 19:53:52,407 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:52,407 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-05 19:53:52,412 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-05 19:53:52,412 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:52,412 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:53:52,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:52,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:53:52,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:53:52,412 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-05 19:53:52,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:53:52,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:53:52,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:52,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:52,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:53:52,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:52,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:52,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,37037,1678046018983': 2023-03-05 19:53:52,416 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-05 19:53:52,416 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-05 19:53:52,416 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-05 19:53:52,416 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-05 19:53:52,416 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:52,416 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-05 19:53:52,418 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:52,418 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:53:52,418 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:52,418 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:52,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:52,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:53:52,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:53:52,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:52,418 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:52,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:53:52,418 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:53:52,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:53:52,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:52,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:52,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:53:52,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:52,420 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:52,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:52,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:53:52,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:52,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:52,423 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:53:52,423 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:53:52,423 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:53:52,423 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:52,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:53:52,423 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:52,423 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-05 19:53:52,424 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-05 19:53:52,423 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:53:52,424 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:52,424 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:52,424 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:52,424 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:52,424 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:53:52,424 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-03-05 19:53:52,424 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:52,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:53:52,424 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-05 19:53:52,424 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-05 19:53:52,524 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-05 19:53:52,526 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-05 19:53:52,531 DEBUG [Listener at localhost/40787] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-03-05 19:53:52,533 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:49532, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-03-05 19:53:52,535 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=38221] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:52,548 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] procedure2.ProcedureExecutor(1029): Stored pid=12, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-03-05 19:53:52,549 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=12, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-03-05 19:53:52,550 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=12, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678046632550 2023-03-05 19:53:52,550 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-05 19:53:52,551 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-05 19:53:52,556 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthly' and column 'family1' 2023-03-05 19:53:52,556 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-05 19:53:52,556 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] zookeeper.ReadOnlyZKClient(139): Connect 0x3ed3bf19 to 127.0.0.1:62957 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-05 19:53:52,561 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@116e4cad, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-05 19:53:52,562 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-05 19:53:53,437 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103afa602934191431bb3c1583c69f5ea15 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20160103afa602934191431bb3c1583c69f5ea15 2023-03-05 19:53:53,438 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-05 19:53:53,456 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-05 19:53:53,458 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:49462, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-05 19:53:53,468 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-05 19:53:53,470 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:55244, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-05 19:53:53,506 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/cdb1304b2eee403ab41322be80e891d1 first=Optional[r3] last=Optional[r6] 2023-03-05 19:53:53,521 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthly,,1678046023317.04217a89b4b7147d2fd1e001ac928c17., hostname=jenkins-hbase4.apache.org,37037,1678046018983, seqNum=2 for row with hfile group [{family1,hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/cdb1304b2eee403ab41322be80e891d1}] 2023-03-05 19:53:53,541 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(763): Validating hfile at hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/cdb1304b2eee403ab41322be80e891d1 for inclusion in 04217a89b4b7147d2fd1e001ac928c17/family1 2023-03-05 19:53:53,549 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-03-05 19:53:53,549 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(777): Region bounds: first= last= 2023-03-05 19:53:53,549 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HRegion(2446): Flush status journal for 04217a89b4b7147d2fd1e001ac928c17: 2023-03-05 19:53:53,550 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/cdb1304b2eee403ab41322be80e891d1 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__u0tu14ln0uu7q141vl9srn637v3cl7dp2e28ttur2knut8n9f1dt7ddri1vm3l40/family1/cdb1304b2eee403ab41322be80e891d1 2023-03-05 19:53:53,555 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__u0tu14ln0uu7q141vl9srn637v3cl7dp2e28ttur2knut8n9f1dt7ddri1vm3l40/family1/cdb1304b2eee403ab41322be80e891d1 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/family1/dd3a94da041f4cdba533a15f4df16c10_SeqId_44_ 2023-03-05 19:53:53,556 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(849): Loaded HFile hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__u0tu14ln0uu7q141vl9srn637v3cl7dp2e28ttur2knut8n9f1dt7ddri1vm3l40/family1/cdb1304b2eee403ab41322be80e891d1 into 04217a89b4b7147d2fd1e001ac928c17/family1 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/family1/dd3a94da041f4cdba533a15f4df16c10_SeqId_44_ - updating store file list. 2023-03-05 19:53:53,563 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(882): Loaded HFile hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/family1/dd3a94da041f4cdba533a15f4df16c10_SeqId_44_ into 04217a89b4b7147d2fd1e001ac928c17/family1 2023-03-05 19:53:53,563 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(855): Successfully loaded hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__u0tu14ln0uu7q141vl9srn637v3cl7dp2e28ttur2knut8n9f1dt7ddri1vm3l40/family1/cdb1304b2eee403ab41322be80e891d1 into 04217a89b4b7147d2fd1e001ac928c17/family1 (new location: hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/family1/dd3a94da041f4cdba533a15f4df16c10_SeqId_44_) 2023-03-05 19:53:53,564 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__u0tu14ln0uu7q141vl9srn637v3cl7dp2e28ttur2knut8n9f1dt7ddri1vm3l40/family1/cdb1304b2eee403ab41322be80e891d1 2023-03-05 19:53:53,566 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-03-05 19:53:53,574 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51357 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-03-05 19:53:53,576 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(1912): 04217a89b4b7147d2fd1e001ac928c17/family1 is initiating minor compaction 2023-03-05 19:53:53,577 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@559d1a1d; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-05 19:53:53,578 INFO [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 04217a89b4b7147d2fd1e001ac928c17/family1 in testMajorCompactionWithWeeklyFollowedByMonthly,,1678046023317.04217a89b4b7147d2fd1e001ac928c17. 2023-03-05 19:53:53,578 INFO [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/family1/6f4e5c4546e84648bb41fcf6383a052a, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/family1/5c6bd7fd0716428c8e350edbf75d0ba5, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/family1/4bd214df671540adb8ab271686620731, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/family1/f8200947e118445cb3e84c45d8c916ef, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/family1/71e1523680814c9c83a78eede5abdedc, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/family1/1e9f737fbd214e2b80fbb6018f5a8195, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/family1/5ceb942597e9436fb85b550b949953ae, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/family1/176651469a5f4bd59e089315da8b846d, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/family1/cb5d9649104f454bbdd20323fccb624b, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/family1/dd3a94da041f4cdba533a15f4df16c10_SeqId_44_] into tmpdir=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/.tmp, totalSize=50.2 K 2023-03-05 19:53:53,579 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.Compactor(207): Compacting 6f4e5c4546e84648bb41fcf6383a052a, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-03-05 19:53:53,579 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.Compactor(207): Compacting 5c6bd7fd0716428c8e350edbf75d0ba5, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=13 2023-03-05 19:53:53,580 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.Compactor(207): Compacting 4bd214df671540adb8ab271686620731, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=17 2023-03-05 19:53:53,580 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.Compactor(207): Compacting f8200947e118445cb3e84c45d8c916ef, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=21 2023-03-05 19:53:53,580 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.Compactor(207): Compacting 71e1523680814c9c83a78eede5abdedc, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=25 2023-03-05 19:53:53,580 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.Compactor(207): Compacting 1e9f737fbd214e2b80fbb6018f5a8195, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=29 2023-03-05 19:53:53,580 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.Compactor(207): Compacting 5ceb942597e9436fb85b550b949953ae, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33 2023-03-05 19:53:53,580 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.Compactor(207): Compacting 176651469a5f4bd59e089315da8b846d, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=37 2023-03-05 19:53:53,580 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.Compactor(207): Compacting cb5d9649104f454bbdd20323fccb624b, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=41 2023-03-05 19:53:53,580 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.Compactor(207): Compacting dd3a94da041f4cdba533a15f4df16c10_SeqId_44_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=45 2023-03-05 19:53:53,597 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-05 19:53:53,599 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-05 19:53:53,606 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015122852779b402a504bd392803354f70b558d to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015122852779b402a504bd392803354f70b558d 2023-03-05 19:53:53,609 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015123119dc85eb575c4d5bbb744cebcb76689d to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015123119dc85eb575c4d5bbb744cebcb76689d 2023-03-05 19:53:53,614 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20160101e004855f93994e5b983bf1dbebd3bcd5 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20160101e004855f93994e5b983bf1dbebd3bcd5 2023-03-05 19:53:53,617 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20160103d216c95bc48b4439ba6c3d0fd1c357ff to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20160103d216c95bc48b4439ba6c3d0fd1c357ff 2023-03-05 19:53:53,617 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthly 4 expired mob files are deleted 2023-03-05 19:53:53,617 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-03-05 19:53:53,620 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-05 19:53:53,620 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-05 19:53:53,620 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-05 19:53:53,624 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-05 19:53:53,624 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-05 19:53:53,624 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-05 19:53:53,624 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-05 19:53:53,624 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-05 19:53:53,658 INFO [RS:0;jenkins-hbase4:37037-shortCompactions-0] throttle.PressureAwareThroughputController(145): 04217a89b4b7147d2fd1e001ac928c17#family1#compaction#10 average throughput is 0.24 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-05 19:53:53,732 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120599ffb4c4114946cf8d3a332392f73a11 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015120599ffb4c4114946cf8d3a332392f73a11 2023-03-05 19:53:53,734 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-05 19:53:53,769 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/d5ff41321335485fa9a0e0408eeb5844 first=Optional[r00] last=Optional[r2] 2023-03-05 19:53:53,770 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthly,,1678046023317.04217a89b4b7147d2fd1e001ac928c17., hostname=jenkins-hbase4.apache.org,37037,1678046018983, seqNum=2 for row with hfile group [{family1,hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/d5ff41321335485fa9a0e0408eeb5844}] 2023-03-05 19:53:53,772 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(763): Validating hfile at hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/d5ff41321335485fa9a0e0408eeb5844 for inclusion in 04217a89b4b7147d2fd1e001ac928c17/family1 2023-03-05 19:53:53,775 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(775): HFile bounds: first=r00 last=r2 2023-03-05 19:53:53,775 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(777): Region bounds: first= last= 2023-03-05 19:53:53,776 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HRegion(2446): Flush status journal for 04217a89b4b7147d2fd1e001ac928c17: 2023-03-05 19:53:53,777 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/d5ff41321335485fa9a0e0408eeb5844 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__kva5gn3krnnjk5hnh49ihm86l2cu4nr8maa9v2f3olfv455q2ka08nkht3j0qcuo/family1/d5ff41321335485fa9a0e0408eeb5844 2023-03-05 19:53:53,780 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__kva5gn3krnnjk5hnh49ihm86l2cu4nr8maa9v2f3olfv455q2ka08nkht3j0qcuo/family1/d5ff41321335485fa9a0e0408eeb5844 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/family1/f4a29bdaf00f468fbe899f88f8319b92_SeqId_46_ 2023-03-05 19:53:53,781 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(849): Loaded HFile hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__kva5gn3krnnjk5hnh49ihm86l2cu4nr8maa9v2f3olfv455q2ka08nkht3j0qcuo/family1/d5ff41321335485fa9a0e0408eeb5844 into 04217a89b4b7147d2fd1e001ac928c17/family1 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/family1/f4a29bdaf00f468fbe899f88f8319b92_SeqId_46_ - updating store file list. 2023-03-05 19:53:53,787 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(882): Loaded HFile hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/family1/f4a29bdaf00f468fbe899f88f8319b92_SeqId_46_ into 04217a89b4b7147d2fd1e001ac928c17/family1 2023-03-05 19:53:53,787 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(855): Successfully loaded hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__kva5gn3krnnjk5hnh49ihm86l2cu4nr8maa9v2f3olfv455q2ka08nkht3j0qcuo/family1/d5ff41321335485fa9a0e0408eeb5844 into 04217a89b4b7147d2fd1e001ac928c17/family1 (new location: hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/family1/f4a29bdaf00f468fbe899f88f8319b92_SeqId_46_) 2023-03-05 19:53:53,787 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__kva5gn3krnnjk5hnh49ihm86l2cu4nr8maa9v2f3olfv455q2ka08nkht3j0qcuo/family1/d5ff41321335485fa9a0e0408eeb5844 2023-03-05 19:53:53,790 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-05 19:53:53,791 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-05 19:53:53,794 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015113022f99052f9f54e4ba7bb9610e5a3d6ab to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015113022f99052f9f54e4ba7bb9610e5a3d6ab 2023-03-05 19:53:53,795 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151201f0e77ddb6a9547aa825b10f096f7a98b to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151201f0e77ddb6a9547aa825b10f096f7a98b 2023-03-05 19:53:53,797 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015120589cd5b0c3d6c4db0958e4fbec7c6a668 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015120589cd5b0c3d6c4db0958e4fbec7c6a668 2023-03-05 19:53:53,797 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthly 3 expired mob files are deleted 2023-03-05 19:53:53,797 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-03-05 19:53:53,797 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-05 19:53:53,797 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-05 19:53:53,798 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-05 19:53:53,799 DEBUG [PEWorker-4] locking.LockProcedure(242): UNLOCKED pid=12, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-03-05 19:53:53,801 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=12, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE in 1.2590 sec 2023-03-05 19:53:53,938 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.HMaster$12(2454): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthly from 'testMajorCompactionWithWeeklyFollowedByMonthly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-03-05 19:53:53,946 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure2.ProcedureExecutor(1029): Stored pid=13, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:54,089 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/.tmp/family1/b48490b0bf3b4b03a7a2189c63751087 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/family1/b48490b0bf3b4b03a7a2189c63751087 2023-03-05 19:53:54,101 INFO [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 04217a89b4b7147d2fd1e001ac928c17/family1 of 04217a89b4b7147d2fd1e001ac928c17 into b48490b0bf3b4b03a7a2189c63751087(size=6.6 K), total size for store is 358.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-05 19:53:54,101 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 04217a89b4b7147d2fd1e001ac928c17: 2023-03-05 19:53:54,103 INFO [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthly,,1678046023317.04217a89b4b7147d2fd1e001ac928c17., storeName=family1, fileCount=10, fileSize=50.2 K (5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.5 K), priority=2, time=1678046033575; duration=0sec 2023-03-05 19:53:54,104 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-05 19:53:54,207 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/family1/6f4e5c4546e84648bb41fcf6383a052a, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/family1/5c6bd7fd0716428c8e350edbf75d0ba5, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/family1/4bd214df671540adb8ab271686620731, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/family1/f8200947e118445cb3e84c45d8c916ef, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/family1/71e1523680814c9c83a78eede5abdedc, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/family1/1e9f737fbd214e2b80fbb6018f5a8195, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/family1/5ceb942597e9436fb85b550b949953ae, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/family1/176651469a5f4bd59e089315da8b846d, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/family1/cb5d9649104f454bbdd20323fccb624b, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/family1/dd3a94da041f4cdba533a15f4df16c10_SeqId_44_] to archive 2023-03-05 19:53:54,209 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-05 19:53:54,211 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/family1/6f4e5c4546e84648bb41fcf6383a052a to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/family1/6f4e5c4546e84648bb41fcf6383a052a 2023-03-05 19:53:54,213 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/family1/5c6bd7fd0716428c8e350edbf75d0ba5 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/family1/5c6bd7fd0716428c8e350edbf75d0ba5 2023-03-05 19:53:54,214 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/family1/4bd214df671540adb8ab271686620731 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/family1/4bd214df671540adb8ab271686620731 2023-03-05 19:53:54,216 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/family1/f8200947e118445cb3e84c45d8c916ef to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/family1/f8200947e118445cb3e84c45d8c916ef 2023-03-05 19:53:54,217 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/family1/71e1523680814c9c83a78eede5abdedc to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/family1/71e1523680814c9c83a78eede5abdedc 2023-03-05 19:53:54,218 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/family1/1e9f737fbd214e2b80fbb6018f5a8195 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/family1/1e9f737fbd214e2b80fbb6018f5a8195 2023-03-05 19:53:54,220 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/family1/5ceb942597e9436fb85b550b949953ae to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/family1/5ceb942597e9436fb85b550b949953ae 2023-03-05 19:53:54,221 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/family1/176651469a5f4bd59e089315da8b846d to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/family1/176651469a5f4bd59e089315da8b846d 2023-03-05 19:53:54,223 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/family1/cb5d9649104f454bbdd20323fccb624b to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/family1/cb5d9649104f454bbdd20323fccb624b 2023-03-05 19:53:54,225 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/family1/dd3a94da041f4cdba533a15f4df16c10_SeqId_44_ to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/family1/dd3a94da041f4cdba533a15f4df16c10_SeqId_44_ 2023-03-05 19:53:54,361 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/.tabledesc/.tableinfo.0000000002 2023-03-05 19:53:54,363 DEBUG [PEWorker-1] util.FSTableDescriptors(519): Deleted hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/.tabledesc/.tableinfo.0000000001 2023-03-05 19:53:54,363 INFO [PEWorker-1] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/.tabledesc/.tableinfo.0000000002 2023-03-05 19:53:54,368 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=14, ppid=13, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly}] 2023-03-05 19:53:54,380 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=15, ppid=14, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=04217a89b4b7147d2fd1e001ac928c17, REOPEN/MOVE}] 2023-03-05 19:53:54,382 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=15, ppid=14, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=04217a89b4b7147d2fd1e001ac928c17, REOPEN/MOVE 2023-03-05 19:53:54,383 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=04217a89b4b7147d2fd1e001ac928c17, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:54,383 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1678046023317.04217a89b4b7147d2fd1e001ac928c17.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1678046034383"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678046034383"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678046034383"}]},"ts":"1678046034383"} 2023-03-05 19:53:54,386 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=16, ppid=15, state=RUNNABLE; CloseRegionProcedure 04217a89b4b7147d2fd1e001ac928c17, server=jenkins-hbase4.apache.org,37037,1678046018983}] 2023-03-05 19:53:54,545 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 04217a89b4b7147d2fd1e001ac928c17 2023-03-05 19:53:54,545 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 04217a89b4b7147d2fd1e001ac928c17, disabling compactions & flushes 2023-03-05 19:53:54,545 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthly,,1678046023317.04217a89b4b7147d2fd1e001ac928c17. 2023-03-05 19:53:54,545 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1678046023317.04217a89b4b7147d2fd1e001ac928c17. 2023-03-05 19:53:54,545 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1678046023317.04217a89b4b7147d2fd1e001ac928c17. after waiting 0 ms 2023-03-05 19:53:54,545 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthly,,1678046023317.04217a89b4b7147d2fd1e001ac928c17. 2023-03-05 19:53:54,553 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/recovered.edits/49.seqid, newMaxSeqId=49, maxSeqId=1 2023-03-05 19:53:54,554 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthly,,1678046023317.04217a89b4b7147d2fd1e001ac928c17. 2023-03-05 19:53:54,555 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 04217a89b4b7147d2fd1e001ac928c17: 2023-03-05 19:53:54,555 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3509): Not adding moved region record: 04217a89b4b7147d2fd1e001ac928c17 to self. 2023-03-05 19:53:54,557 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 04217a89b4b7147d2fd1e001ac928c17 2023-03-05 19:53:54,558 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=04217a89b4b7147d2fd1e001ac928c17, regionState=CLOSED 2023-03-05 19:53:54,558 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1678046023317.04217a89b4b7147d2fd1e001ac928c17.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1678046034558"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678046034558"}]},"ts":"1678046034558"} 2023-03-05 19:53:54,562 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=16, resume processing ppid=15 2023-03-05 19:53:54,562 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=16, ppid=15, state=SUCCESS; CloseRegionProcedure 04217a89b4b7147d2fd1e001ac928c17, server=jenkins-hbase4.apache.org,37037,1678046018983 in 174 msec 2023-03-05 19:53:54,563 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=15, ppid=14, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=04217a89b4b7147d2fd1e001ac928c17, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,37037,1678046018983; forceNewPlan=false, retain=true 2023-03-05 19:53:54,713 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=04217a89b4b7147d2fd1e001ac928c17, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:54,714 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1678046023317.04217a89b4b7147d2fd1e001ac928c17.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1678046034713"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678046034713"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678046034713"}]},"ts":"1678046034713"} 2023-03-05 19:53:54,716 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=17, ppid=15, state=RUNNABLE; OpenRegionProcedure 04217a89b4b7147d2fd1e001ac928c17, server=jenkins-hbase4.apache.org,37037,1678046018983}] 2023-03-05 19:53:54,873 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthly,,1678046023317.04217a89b4b7147d2fd1e001ac928c17. 2023-03-05 19:53:54,873 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 04217a89b4b7147d2fd1e001ac928c17, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthly,,1678046023317.04217a89b4b7147d2fd1e001ac928c17.', STARTKEY => '', ENDKEY => ''} 2023-03-05 19:53:54,873 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthly 04217a89b4b7147d2fd1e001ac928c17 2023-03-05 19:53:54,873 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthly,,1678046023317.04217a89b4b7147d2fd1e001ac928c17.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-05 19:53:54,873 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 04217a89b4b7147d2fd1e001ac928c17 2023-03-05 19:53:54,873 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 04217a89b4b7147d2fd1e001ac928c17 2023-03-05 19:53:54,875 INFO [StoreOpener-04217a89b4b7147d2fd1e001ac928c17-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 04217a89b4b7147d2fd1e001ac928c17 2023-03-05 19:53:54,876 DEBUG [StoreOpener-04217a89b4b7147d2fd1e001ac928c17-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/family1 2023-03-05 19:53:54,876 DEBUG [StoreOpener-04217a89b4b7147d2fd1e001ac928c17-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/family1 2023-03-05 19:53:54,877 INFO [StoreOpener-04217a89b4b7147d2fd1e001ac928c17-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 04217a89b4b7147d2fd1e001ac928c17 columnFamilyName family1 2023-03-05 19:53:54,884 DEBUG [StoreOpener-04217a89b4b7147d2fd1e001ac928c17-1] regionserver.HStore(539): loaded hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/family1/b48490b0bf3b4b03a7a2189c63751087 2023-03-05 19:53:54,890 DEBUG [StoreOpener-04217a89b4b7147d2fd1e001ac928c17-1] regionserver.HStore(539): loaded hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/family1/bdb3cf4515f244fb8f09e1ed853027db 2023-03-05 19:53:54,895 DEBUG [StoreOpener-04217a89b4b7147d2fd1e001ac928c17-1] regionserver.HStore(539): loaded hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/family1/f4a29bdaf00f468fbe899f88f8319b92_SeqId_46_ 2023-03-05 19:53:54,896 INFO [StoreOpener-04217a89b4b7147d2fd1e001ac928c17-1] regionserver.HStore(310): Store=04217a89b4b7147d2fd1e001ac928c17/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-05 19:53:54,897 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17 2023-03-05 19:53:54,899 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17 2023-03-05 19:53:54,902 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 04217a89b4b7147d2fd1e001ac928c17 2023-03-05 19:53:54,903 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 04217a89b4b7147d2fd1e001ac928c17; next sequenceid=50; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72054719, jitterRate=0.0736989825963974}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-05 19:53:54,903 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 04217a89b4b7147d2fd1e001ac928c17: 2023-03-05 19:53:54,904 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthly,,1678046023317.04217a89b4b7147d2fd1e001ac928c17., pid=17, masterSystemTime=1678046034869 2023-03-05 19:53:54,904 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-05 19:53:54,905 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-03-05 19:53:54,907 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 367249 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-03-05 19:53:54,907 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.HStore(1912): 04217a89b4b7147d2fd1e001ac928c17/family1 is initiating minor compaction (all files) 2023-03-05 19:53:54,907 INFO [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 04217a89b4b7147d2fd1e001ac928c17/family1 in testMajorCompactionWithWeeklyFollowedByMonthly,,1678046023317.04217a89b4b7147d2fd1e001ac928c17. 2023-03-05 19:53:54,907 INFO [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/family1/bdb3cf4515f244fb8f09e1ed853027db, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/family1/b48490b0bf3b4b03a7a2189c63751087, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/family1/f4a29bdaf00f468fbe899f88f8319b92_SeqId_46_] into tmpdir=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/.tmp, totalSize=358.6 K 2023-03-05 19:53:54,907 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthly,,1678046023317.04217a89b4b7147d2fd1e001ac928c17. 2023-03-05 19:53:54,907 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthly,,1678046023317.04217a89b4b7147d2fd1e001ac928c17. 2023-03-05 19:53:54,908 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.Compactor(207): Compacting bdb3cf4515f244fb8f09e1ed853027db, keycount=1000, bloomtype=ROW, size=175.8 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878843274 2023-03-05 19:53:54,908 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=04217a89b4b7147d2fd1e001ac928c17, regionState=OPEN, openSeqNum=50, regionLocation=jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:54,908 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1678046023317.04217a89b4b7147d2fd1e001ac928c17.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1678046034908"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678046034908"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678046034908"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678046034908"}]},"ts":"1678046034908"} 2023-03-05 19:53:54,908 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.Compactor(207): Compacting b48490b0bf3b4b03a7a2189c63751087, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621243274 2023-03-05 19:53:54,909 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.Compactor(207): Compacting f4a29bdaf00f468fbe899f88f8319b92_SeqId_46_, keycount=1002, bloomtype=ROW, size=176.2 K, encoding=NONE, compression=NONE, seqNum=47, earliestPutTs=1448878843274 2023-03-05 19:53:54,912 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=17, resume processing ppid=15 2023-03-05 19:53:54,912 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=17, ppid=15, state=SUCCESS; OpenRegionProcedure 04217a89b4b7147d2fd1e001ac928c17, server=jenkins-hbase4.apache.org,37037,1678046018983 in 194 msec 2023-03-05 19:53:54,914 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=15, resume processing ppid=14 2023-03-05 19:53:54,914 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=15, ppid=14, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=04217a89b4b7147d2fd1e001ac928c17, REOPEN/MOVE in 532 msec 2023-03-05 19:53:54,918 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=14, resume processing ppid=13 2023-03-05 19:53:54,918 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=14, ppid=13, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly in 549 msec 2023-03-05 19:53:54,921 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=13, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly in 978 msec 2023-03-05 19:53:54,924 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1227): Checking to see if procedure is done pid=13 2023-03-05 19:53:54,932 INFO [Listener at localhost/40787] compactions.TestMobCompactor(278): alter status finished 2023-03-05 19:53:54,933 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=38221] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-05 19:53:54,935 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] procedure2.ProcedureExecutor(1029): Stored pid=18, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-03-05 19:53:54,936 INFO [RS:0;jenkins-hbase4:37037-shortCompactions-0] throttle.PressureAwareThroughputController(145): 04217a89b4b7147d2fd1e001ac928c17#family1#compaction#11 average throughput is 9.18 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-05 19:53:54,941 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=18, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-03-05 19:53:54,943 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=18, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678046634943 2023-03-05 19:53:54,943 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-05 19:53:54,944 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-05 19:53:54,944 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 5 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthly' and column 'family1' 2023-03-05 19:53:54,944 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-05 19:53:54,945 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] zookeeper.ReadOnlyZKClient(139): Connect 0x32fab577 to 127.0.0.1:62957 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-05 19:53:54,960 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@183dd29, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-05 19:53:54,961 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-05 19:53:54,961 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-05 19:53:54,961 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-05 19:53:54,961 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-05 19:53:54,961 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-05 19:53:55,799 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511281823a3d6496c4c0e88cc5a5b6c5a8971 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201511281823a3d6496c4c0e88cc5a5b6c5a8971 2023-03-05 19:53:55,800 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-05 19:53:55,813 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-05 19:53:55,814 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:49470, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-05 19:53:55,822 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-05 19:53:55,824 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:55250, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-05 19:53:55,836 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/a08f5de23ef1491787b39716c99796b6 first=Optional[r05] last=Optional[r8] 2023-03-05 19:53:55,837 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthly,,1678046023317.04217a89b4b7147d2fd1e001ac928c17., hostname=jenkins-hbase4.apache.org,37037,1678046018983, seqNum=50 for row with hfile group [{family1,hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/a08f5de23ef1491787b39716c99796b6}] 2023-03-05 19:53:55,843 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(763): Validating hfile at hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/a08f5de23ef1491787b39716c99796b6 for inclusion in 04217a89b4b7147d2fd1e001ac928c17/family1 2023-03-05 19:53:55,847 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(775): HFile bounds: first=r05 last=r8 2023-03-05 19:53:55,847 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(777): Region bounds: first= last= 2023-03-05 19:53:55,847 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HRegion(2446): Flush status journal for 04217a89b4b7147d2fd1e001ac928c17: 2023-03-05 19:53:55,848 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/a08f5de23ef1491787b39716c99796b6 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__vmkhf1cp1d2giselvnuelssjuu5257ffj8qur02rv0ok0o51l1cohof653kmh5p9/family1/a08f5de23ef1491787b39716c99796b6 2023-03-05 19:53:55,850 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__vmkhf1cp1d2giselvnuelssjuu5257ffj8qur02rv0ok0o51l1cohof653kmh5p9/family1/a08f5de23ef1491787b39716c99796b6 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/family1/7fbe6306b2aa490ca521d53a905a5781_SeqId_52_ 2023-03-05 19:53:55,851 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(849): Loaded HFile hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__vmkhf1cp1d2giselvnuelssjuu5257ffj8qur02rv0ok0o51l1cohof653kmh5p9/family1/a08f5de23ef1491787b39716c99796b6 into 04217a89b4b7147d2fd1e001ac928c17/family1 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/family1/7fbe6306b2aa490ca521d53a905a5781_SeqId_52_ - updating store file list. 2023-03-05 19:53:55,856 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(882): Loaded HFile hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/family1/7fbe6306b2aa490ca521d53a905a5781_SeqId_52_ into 04217a89b4b7147d2fd1e001ac928c17/family1 2023-03-05 19:53:55,856 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(855): Successfully loaded hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__vmkhf1cp1d2giselvnuelssjuu5257ffj8qur02rv0ok0o51l1cohof653kmh5p9/family1/a08f5de23ef1491787b39716c99796b6 into 04217a89b4b7147d2fd1e001ac928c17/family1 (new location: hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/family1/7fbe6306b2aa490ca521d53a905a5781_SeqId_52_) 2023-03-05 19:53:55,856 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__vmkhf1cp1d2giselvnuelssjuu5257ffj8qur02rv0ok0o51l1cohof653kmh5p9/family1/a08f5de23ef1491787b39716c99796b6 2023-03-05 19:53:55,859 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-05 19:53:55,859 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-05 19:53:55,861 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151120775a3a7933f0436da2e681e1c2e446a8 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151120775a3a7933f0436da2e681e1c2e446a8 2023-03-05 19:53:55,862 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201511280b2d7d49b36348a2aae676dbd9d7faf6 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201511280b2d7d49b36348a2aae676dbd9d7faf6 2023-03-05 19:53:55,863 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthly 2 expired mob files are deleted 2023-03-05 19:53:55,863 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-03-05 19:53:55,863 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-05 19:53:55,863 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-05 19:53:55,863 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-05 19:53:55,863 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-05 19:53:55,863 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-05 19:53:55,864 DEBUG [PEWorker-3] locking.LockProcedure(242): UNLOCKED pid=18, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-03-05 19:53:55,866 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=18, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE in 930 msec 2023-03-05 19:53:55,968 INFO [Listener at localhost/40787] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthly Thread=250 (was 211) Potentially hanging thread: (jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:62957@0x3ed3bf19-SendThread(127.0.0.1:62957) sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223) Potentially hanging thread: hfile_cleaner-dir-scan-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: htable-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-4-3 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x44d2b9a5-metaLookup-shared--pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_260397064_15 at /127.0.0.1:36626 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x155052e5-metaLookup-shared--pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_260397064_15 at /127.0.0.1:36586 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_260397064_15 at /127.0.0.1:36596 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_395068685_15 at /127.0.0.1:55566 [Waiting for operation #16] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_260397064_15 at /127.0.0.1:36654 [Waiting for operation #15] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_260397064_15 at /127.0.0.1:36718 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_260397064_15 at /127.0.0.1:36664 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataStreamer for file /user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/.tmp/family1/228dc4de81b0405c8f63928691a04326 java.lang.Object.wait(Native Method) org.apache.hadoop.hdfs.DataStreamer.run(DataStreamer.java:678) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_260397064_15 at /127.0.0.1:36714 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_260397064_15 at /127.0.0.1:55614 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x44d2b9a5-shared-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: MobCompactor-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-1-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-4-2 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hfile_cleaner-dir-scan-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:62957@0x3ed3bf19 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$45/715920721.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:0;jenkins-hbase4:37037-shortCompactions-0 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.DFSOutputStream.completeFile(DFSOutputStream.java:940) org.apache.hadoop.hdfs.DFSOutputStream.completeFile(DFSOutputStream.java:880) org.apache.hadoop.hdfs.DFSOutputStream.closeImpl(DFSOutputStream.java:863) org.apache.hadoop.hdfs.DFSOutputStream.close(DFSOutputStream.java:818) org.apache.hadoop.fs.FSDataOutputStream$PositionCache.close(FSDataOutputStream.java:72) org.apache.hadoop.fs.FSDataOutputStream.close(FSDataOutputStream.java:106) org.apache.hadoop.hbase.io.hfile.HFileWriterImpl.finishClose(HFileWriterImpl.java:820) org.apache.hadoop.hbase.io.hfile.HFileWriterImpl.close(HFileWriterImpl.java:654) org.apache.hadoop.hbase.regionserver.StoreFileWriter.close(StoreFileWriter.java:359) org.apache.hadoop.hbase.regionserver.compactions.DefaultCompactor.commitWriter(DefaultCompactor.java:89) org.apache.hadoop.hbase.regionserver.compactions.DefaultCompactor.commitWriter(DefaultCompactor.java:42) org.apache.hadoop.hbase.regionserver.compactions.Compactor.compact(Compactor.java:362) org.apache.hadoop.hbase.mob.DefaultMobStoreCompactor.compact(DefaultMobStoreCompactor.java:108) org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext.compact(DefaultStoreEngine.java:122) org.apache.hadoop.hbase.regionserver.HStore.compact(HStore.java:1473) org.apache.hadoop.hbase.regionserver.HRegion.compact(HRegion.java:2267) org.apache.hadoop.hbase.regionserver.CompactSplit$CompactionRunner.doCompaction(CompactSplit.java:625) org.apache.hadoop.hbase.regionserver.CompactSplit$CompactionRunner.run(CompactSplit.java:673) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_260397064_15 at /127.0.0.1:55600 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Timer for 'HBase' metrics system java.lang.Object.wait(Native Method) java.util.TimerThread.mainLoop(Timer.java:552) java.util.TimerThread.run(Timer.java:505) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:62957@0x32fab577-SendThread(127.0.0.1:62957) sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223) Potentially hanging thread: hconnection-0x155052e5-shared-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x155052e5-shared-pool-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x155052e5-shared-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_260397064_15 at /127.0.0.1:36640 [Waiting for operation #19] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_260397064_15 at /127.0.0.1:55588 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RSProcedureDispatcher-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x44d2b9a5-shared-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:62957@0x32fab577-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_260397064_15 at /127.0.0.1:36612 [Waiting for operation #13] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:62957@0x32fab577 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$45/715920721.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:62957@0x3ed3bf19-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: RPCClient-NioEventLoopGroup-4-4 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_260397064_15 at /127.0.0.1:55640 [Waiting for operation #24] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x155052e5-shared-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=586 (was 475) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=327 (was 361), ProcessCount=171 (was 171), AvailableMemoryMB=4937 (was 5259) 2023-03-05 19:53:55,980 INFO [Listener at localhost/40787] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithMonthlyPolicy Thread=250, OpenFileDescriptor=586, MaxFileDescriptor=60000, SystemLoadAverage=327, ProcessCount=171, AvailableMemoryMB=4936 2023-03-05 19:53:55,983 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithMonthlyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-03-05 19:53:55,984 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure2.ProcedureExecutor(1029): Stored pid=19, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy 2023-03-05 19:53:55,986 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_PRE_OPERATION 2023-03-05 19:53:55,986 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithMonthlyPolicy" procId is: 19 2023-03-05 19:53:55,987 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-05 19:53:55,987 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-03-05 19:53:55,989 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/.tmp/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec 2023-03-05 19:53:55,989 DEBUG [HFileArchiver-3] backup.HFileArchiver(153): Directory hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/.tmp/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec empty. 2023-03-05 19:53:55,990 DEBUG [HFileArchiver-3] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/.tmp/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec 2023-03-05 19:53:55,990 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithMonthlyPolicy regions 2023-03-05 19:53:56,089 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-03-05 19:53:56,177 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/.tmp/family1/228dc4de81b0405c8f63928691a04326 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/family1/228dc4de81b0405c8f63928691a04326 2023-03-05 19:53:56,183 INFO [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 04217a89b4b7147d2fd1e001ac928c17/family1 of 04217a89b4b7147d2fd1e001ac928c17 into 228dc4de81b0405c8f63928691a04326(size=177.2 K), total size for store is 182.4 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-05 19:53:56,183 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 04217a89b4b7147d2fd1e001ac928c17: 2023-03-05 19:53:56,183 INFO [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testMajorCompactionWithWeeklyFollowedByMonthly,,1678046023317.04217a89b4b7147d2fd1e001ac928c17., storeName=04217a89b4b7147d2fd1e001ac928c17/family1, priority=13, startTime=1678046034904; duration=1sec 2023-03-05 19:53:56,183 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-05 19:53:56,204 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/family1/bdb3cf4515f244fb8f09e1ed853027db, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/family1/b48490b0bf3b4b03a7a2189c63751087, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/family1/f4a29bdaf00f468fbe899f88f8319b92_SeqId_46_] to archive 2023-03-05 19:53:56,205 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-05 19:53:56,207 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/family1/bdb3cf4515f244fb8f09e1ed853027db to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/family1/bdb3cf4515f244fb8f09e1ed853027db 2023-03-05 19:53:56,208 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/family1/b48490b0bf3b4b03a7a2189c63751087 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/family1/b48490b0bf3b4b03a7a2189c63751087 2023-03-05 19:53:56,210 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/family1/f4a29bdaf00f468fbe899f88f8319b92_SeqId_46_ to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/family1/f4a29bdaf00f468fbe899f88f8319b92_SeqId_46_ 2023-03-05 19:53:56,290 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-03-05 19:53:56,400 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/.tmp/data/default/testMajorCompactionWithMonthlyPolicy/.tabledesc/.tableinfo.0000000001 2023-03-05 19:53:56,402 INFO [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(7675): creating {ENCODED => aa3a9f808cf1db1e0667a9fff093c3ec, NAME => 'testMajorCompactionWithMonthlyPolicy,,1678046035983.aa3a9f808cf1db1e0667a9fff093c3ec.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithMonthlyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}, regionDir=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/.tmp 2023-03-05 19:53:56,592 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-03-05 19:53:56,810 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithMonthlyPolicy,,1678046035983.aa3a9f808cf1db1e0667a9fff093c3ec.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-05 19:53:56,810 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1604): Closing aa3a9f808cf1db1e0667a9fff093c3ec, disabling compactions & flushes 2023-03-05 19:53:56,810 INFO [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithMonthlyPolicy,,1678046035983.aa3a9f808cf1db1e0667a9fff093c3ec. 2023-03-05 19:53:56,810 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithMonthlyPolicy,,1678046035983.aa3a9f808cf1db1e0667a9fff093c3ec. 2023-03-05 19:53:56,810 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithMonthlyPolicy,,1678046035983.aa3a9f808cf1db1e0667a9fff093c3ec. after waiting 0 ms 2023-03-05 19:53:56,810 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithMonthlyPolicy,,1678046035983.aa3a9f808cf1db1e0667a9fff093c3ec. 2023-03-05 19:53:56,810 INFO [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1838): Closed testMajorCompactionWithMonthlyPolicy,,1678046035983.aa3a9f808cf1db1e0667a9fff093c3ec. 2023-03-05 19:53:56,811 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1558): Region close journal for aa3a9f808cf1db1e0667a9fff093c3ec: 2023-03-05 19:53:56,814 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_ADD_TO_META 2023-03-05 19:53:56,814 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithMonthlyPolicy,,1678046035983.aa3a9f808cf1db1e0667a9fff093c3ec.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1678046036814"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678046036814"}]},"ts":"1678046036814"} 2023-03-05 19:53:56,816 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-03-05 19:53:56,817 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-05 19:53:56,817 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithMonthlyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678046036817"}]},"ts":"1678046036817"} 2023-03-05 19:53:56,819 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithMonthlyPolicy, state=ENABLING in hbase:meta 2023-03-05 19:53:56,822 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithMonthlyPolicy, region=aa3a9f808cf1db1e0667a9fff093c3ec, ASSIGN}] 2023-03-05 19:53:56,823 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithMonthlyPolicy, region=aa3a9f808cf1db1e0667a9fff093c3ec, ASSIGN 2023-03-05 19:53:56,824 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithMonthlyPolicy, region=aa3a9f808cf1db1e0667a9fff093c3ec, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,37037,1678046018983; forceNewPlan=false, retain=false 2023-03-05 19:53:56,975 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=aa3a9f808cf1db1e0667a9fff093c3ec, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:56,976 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithMonthlyPolicy,,1678046035983.aa3a9f808cf1db1e0667a9fff093c3ec.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1678046036975"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678046036975"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678046036975"}]},"ts":"1678046036975"} 2023-03-05 19:53:56,978 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=21, ppid=20, state=RUNNABLE; OpenRegionProcedure aa3a9f808cf1db1e0667a9fff093c3ec, server=jenkins-hbase4.apache.org,37037,1678046018983}] 2023-03-05 19:53:57,093 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-03-05 19:53:57,133 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithMonthlyPolicy,,1678046035983.aa3a9f808cf1db1e0667a9fff093c3ec. 2023-03-05 19:53:57,134 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => aa3a9f808cf1db1e0667a9fff093c3ec, NAME => 'testMajorCompactionWithMonthlyPolicy,,1678046035983.aa3a9f808cf1db1e0667a9fff093c3ec.', STARTKEY => '', ENDKEY => ''} 2023-03-05 19:53:57,134 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithMonthlyPolicy aa3a9f808cf1db1e0667a9fff093c3ec 2023-03-05 19:53:57,134 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithMonthlyPolicy,,1678046035983.aa3a9f808cf1db1e0667a9fff093c3ec.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-05 19:53:57,134 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for aa3a9f808cf1db1e0667a9fff093c3ec 2023-03-05 19:53:57,134 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for aa3a9f808cf1db1e0667a9fff093c3ec 2023-03-05 19:53:57,136 INFO [StoreOpener-aa3a9f808cf1db1e0667a9fff093c3ec-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region aa3a9f808cf1db1e0667a9fff093c3ec 2023-03-05 19:53:57,137 DEBUG [StoreOpener-aa3a9f808cf1db1e0667a9fff093c3ec-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec/family1 2023-03-05 19:53:57,137 DEBUG [StoreOpener-aa3a9f808cf1db1e0667a9fff093c3ec-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec/family1 2023-03-05 19:53:57,138 INFO [StoreOpener-aa3a9f808cf1db1e0667a9fff093c3ec-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region aa3a9f808cf1db1e0667a9fff093c3ec columnFamilyName family1 2023-03-05 19:53:57,139 INFO [StoreOpener-aa3a9f808cf1db1e0667a9fff093c3ec-1] regionserver.HStore(310): Store=aa3a9f808cf1db1e0667a9fff093c3ec/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-05 19:53:57,140 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec 2023-03-05 19:53:57,141 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec 2023-03-05 19:53:57,144 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for aa3a9f808cf1db1e0667a9fff093c3ec 2023-03-05 19:53:57,146 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-05 19:53:57,147 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened aa3a9f808cf1db1e0667a9fff093c3ec; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66490869, jitterRate=-0.009208843111991882}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-05 19:53:57,147 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for aa3a9f808cf1db1e0667a9fff093c3ec: 2023-03-05 19:53:57,148 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithMonthlyPolicy,,1678046035983.aa3a9f808cf1db1e0667a9fff093c3ec., pid=21, masterSystemTime=1678046037130 2023-03-05 19:53:57,150 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithMonthlyPolicy,,1678046035983.aa3a9f808cf1db1e0667a9fff093c3ec. 2023-03-05 19:53:57,150 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithMonthlyPolicy,,1678046035983.aa3a9f808cf1db1e0667a9fff093c3ec. 2023-03-05 19:53:57,151 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=aa3a9f808cf1db1e0667a9fff093c3ec, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:57,151 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithMonthlyPolicy,,1678046035983.aa3a9f808cf1db1e0667a9fff093c3ec.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1678046037151"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678046037151"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678046037151"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678046037151"}]},"ts":"1678046037151"} 2023-03-05 19:53:57,155 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=21, resume processing ppid=20 2023-03-05 19:53:57,155 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=21, ppid=20, state=SUCCESS; OpenRegionProcedure aa3a9f808cf1db1e0667a9fff093c3ec, server=jenkins-hbase4.apache.org,37037,1678046018983 in 175 msec 2023-03-05 19:53:57,157 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=20, resume processing ppid=19 2023-03-05 19:53:57,157 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=20, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithMonthlyPolicy, region=aa3a9f808cf1db1e0667a9fff093c3ec, ASSIGN in 333 msec 2023-03-05 19:53:57,158 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-05 19:53:57,158 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithMonthlyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678046037158"}]},"ts":"1678046037158"} 2023-03-05 19:53:57,160 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithMonthlyPolicy, state=ENABLED in hbase:meta 2023-03-05 19:53:57,163 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_POST_OPERATION 2023-03-05 19:53:57,165 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=19, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy in 1.1800 sec 2023-03-05 19:53:58,095 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-03-05 19:53:58,095 INFO [Listener at localhost/40787] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithMonthlyPolicy, procId: 19 completed 2023-03-05 19:53:58,129 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-05 19:53:58,135 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-05 19:53:58,135 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-05 19:53:58,135 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:53:58,136 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-05 19:53:58,136 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-05 19:53:58,137 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:53:58,137 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:53:58,138 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:58,138 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:53:58,138 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:53:58,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:53:58,139 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:58,139 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-05 19:53:58,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:53:58,139 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:53:58,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-05 19:53:58,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:53:58,140 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-05 19:53:58,140 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-05 19:53:58,140 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-05 19:53:58,140 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:53:58,141 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-05 19:53:58,142 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-05 19:53:58,142 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-05 19:53:58,142 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1678046035983.aa3a9f808cf1db1e0667a9fff093c3ec. 2023-03-05 19:53:58,143 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1678046035983.aa3a9f808cf1db1e0667a9fff093c3ec. started... 2023-03-05 19:53:58,143 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing aa3a9f808cf1db1e0667a9fff093c3ec 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-03-05 19:53:58,587 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130bc06fd1310204b00aa8ed087fb496dba to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151130bc06fd1310204b00aa8ed087fb496dba 2023-03-05 19:53:58,992 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec/.tmp/family1/bc58a3df1dd0415fbe3b9606b4318973 2023-03-05 19:53:58,999 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec/.tmp/family1/bc58a3df1dd0415fbe3b9606b4318973 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec/family1/bc58a3df1dd0415fbe3b9606b4318973 2023-03-05 19:53:59,004 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec/family1/bc58a3df1dd0415fbe3b9606b4318973, entries=1000, sequenceid=5, filesize=166.0 K 2023-03-05 19:53:59,005 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for aa3a9f808cf1db1e0667a9fff093c3ec in 862ms, sequenceid=5, compaction requested=false 2023-03-05 19:53:59,005 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithMonthlyPolicy' 2023-03-05 19:53:59,006 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for aa3a9f808cf1db1e0667a9fff093c3ec: 2023-03-05 19:53:59,006 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1678046035983.aa3a9f808cf1db1e0667a9fff093c3ec. 2023-03-05 19:53:59,006 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-05 19:53:59,006 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-05 19:53:59,006 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:59,006 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-05 19:53:59,006 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-05 19:53:59,008 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:59,008 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:53:59,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:59,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:53:59,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:53:59,008 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:53:59,009 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-05 19:53:59,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:53:59,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:53:59,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-05 19:53:59,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:59,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:53:59,010 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-05 19:53:59,010 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@674a3074[Count = 0] remaining members to acquire global barrier 2023-03-05 19:53:59,010 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-05 19:53:59,010 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:53:59,011 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:53:59,011 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:53:59,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:53:59,011 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-05 19:53:59,011 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-05 19:53:59,011 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:59,011 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-05 19:53:59,011 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,37037,1678046018983' in zk 2023-03-05 19:53:59,013 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:59,013 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-05 19:53:59,013 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:53:59,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:59,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:53:59,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:53:59,013 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-05 19:53:59,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:53:59,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:53:59,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-05 19:53:59,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:59,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:53:59,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-05 19:53:59,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:59,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,37037,1678046018983': 2023-03-05 19:53:59,016 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-05 19:53:59,016 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-05 19:53:59,016 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-05 19:53:59,016 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-05 19:53:59,016 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-05 19:53:59,016 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-05 19:53:59,020 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:53:59,020 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:53:59,020 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:53:59,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:53:59,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:53:59,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:53:59,020 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:53:59,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:53:59,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:53:59,020 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:53:59,021 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:59,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:53:59,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-05 19:53:59,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:53:59,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:53:59,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-05 19:53:59,022 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:59,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:59,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:53:59,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-05 19:53:59,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:59,024 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:59,024 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:53:59,024 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:53:59,024 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:53:59,024 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:53:59,024 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:53:59,025 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:59,024 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-05 19:53:59,025 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-05 19:53:59,024 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:53:59,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:53:59,025 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:53:59,025 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:53:59,025 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-05 19:53:59,026 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-05 19:53:59,026 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:53:59,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:53:59,025 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:53:59,126 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-05 19:53:59,127 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-05 19:53:59,132 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-05 19:53:59,134 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-05 19:53:59,134 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-05 19:53:59,134 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-05 19:53:59,134 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:53:59,135 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-05 19:53:59,135 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-05 19:53:59,135 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:53:59,135 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:53:59,137 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:53:59,137 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:59,137 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:53:59,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:53:59,137 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:59,137 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-05 19:53:59,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:53:59,138 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:53:59,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-05 19:53:59,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:53:59,138 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-05 19:53:59,138 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-03-05 19:53:59,138 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-05 19:53:59,138 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-05 19:53:59,139 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:53:59,139 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-05 19:53:59,139 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-05 19:53:59,139 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-05 19:53:59,139 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1678046035983.aa3a9f808cf1db1e0667a9fff093c3ec. 2023-03-05 19:53:59,139 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1678046035983.aa3a9f808cf1db1e0667a9fff093c3ec. started... 2023-03-05 19:53:59,139 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing aa3a9f808cf1db1e0667a9fff093c3ec 1/1 column families, dataSize=74 B heapSize=408 B 2023-03-05 19:53:59,554 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128fbc24177ef654af4bae3a984ed76b29f to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151128fbc24177ef654af4bae3a984ed76b29f 2023-03-05 19:53:59,560 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec/.tmp/family1/194ecb5ba7cd44fa9349d80c8ca881fe 2023-03-05 19:53:59,567 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec/.tmp/family1/194ecb5ba7cd44fa9349d80c8ca881fe as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec/family1/194ecb5ba7cd44fa9349d80c8ca881fe 2023-03-05 19:53:59,574 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec/family1/194ecb5ba7cd44fa9349d80c8ca881fe, entries=1, sequenceid=9, filesize=5.0 K 2023-03-05 19:53:59,575 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for aa3a9f808cf1db1e0667a9fff093c3ec in 436ms, sequenceid=9, compaction requested=false 2023-03-05 19:53:59,575 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for aa3a9f808cf1db1e0667a9fff093c3ec: 2023-03-05 19:53:59,575 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1678046035983.aa3a9f808cf1db1e0667a9fff093c3ec. 2023-03-05 19:53:59,576 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-05 19:53:59,576 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-05 19:53:59,576 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:59,576 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-05 19:53:59,576 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-05 19:53:59,578 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:59,578 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:53:59,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:59,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:53:59,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:53:59,578 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:53:59,578 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-05 19:53:59,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:53:59,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:53:59,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-05 19:53:59,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:59,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:53:59,580 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-05 19:53:59,580 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@34e2a227[Count = 0] remaining members to acquire global barrier 2023-03-05 19:53:59,580 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-05 19:53:59,580 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:53:59,583 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:53:59,584 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:53:59,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:53:59,584 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-05 19:53:59,584 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-05 19:53:59,584 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,37037,1678046018983' in zk 2023-03-05 19:53:59,584 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:59,584 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-05 19:53:59,586 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:59,586 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-05 19:53:59,586 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:53:59,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:59,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:53:59,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:53:59,586 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-05 19:53:59,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:53:59,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:53:59,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-05 19:53:59,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:59,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:53:59,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-05 19:53:59,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:59,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,37037,1678046018983': 2023-03-05 19:53:59,590 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-05 19:53:59,590 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-05 19:53:59,590 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-05 19:53:59,590 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-05 19:53:59,590 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-05 19:53:59,590 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-05 19:53:59,592 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:53:59,592 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:53:59,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:53:59,592 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:53:59,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:53:59,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:53:59,592 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:53:59,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:53:59,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:53:59,593 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:53:59,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:53:59,593 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:59,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-05 19:53:59,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:53:59,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:53:59,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-05 19:53:59,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:59,594 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:59,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:53:59,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-05 19:53:59,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:59,601 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:59,601 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:53:59,601 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:53:59,601 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:53:59,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:53:59,601 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:53:59,601 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-05 19:53:59,601 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-05 19:53:59,601 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:53:59,601 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:53:59,602 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:59,602 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:53:59,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:53:59,602 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:53:59,602 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-05 19:53:59,602 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-05 19:53:59,602 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:53:59,603 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:53:59,703 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-05 19:53:59,704 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-05 19:53:59,708 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-05 19:53:59,710 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-05 19:53:59,710 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-05 19:53:59,710 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-05 19:53:59,710 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:53:59,711 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-05 19:53:59,711 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-05 19:53:59,711 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:53:59,711 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:53:59,712 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:59,712 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:53:59,713 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:53:59,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:53:59,713 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:53:59,713 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-05 19:53:59,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:53:59,713 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:53:59,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-05 19:53:59,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:53:59,714 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-05 19:53:59,714 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-03-05 19:53:59,714 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-05 19:53:59,714 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-05 19:53:59,714 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:53:59,714 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-05 19:53:59,715 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-05 19:53:59,715 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-05 19:53:59,715 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1678046035983.aa3a9f808cf1db1e0667a9fff093c3ec. 2023-03-05 19:53:59,715 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1678046035983.aa3a9f808cf1db1e0667a9fff093c3ec. started... 2023-03-05 19:53:59,715 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing aa3a9f808cf1db1e0667a9fff093c3ec 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-05 19:54:00,089 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-05 19:54:00,130 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120140465371c4bb4199aeab0076c6eba76f to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015120140465371c4bb4199aeab0076c6eba76f 2023-03-05 19:54:00,536 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec/.tmp/family1/d2a282c260cb49e5b461a2e98c567b19 2023-03-05 19:54:00,542 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec/.tmp/family1/d2a282c260cb49e5b461a2e98c567b19 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec/family1/d2a282c260cb49e5b461a2e98c567b19 2023-03-05 19:54:00,547 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec/family1/d2a282c260cb49e5b461a2e98c567b19, entries=1, sequenceid=13, filesize=4.9 K 2023-03-05 19:54:00,548 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for aa3a9f808cf1db1e0667a9fff093c3ec in 833ms, sequenceid=13, compaction requested=true 2023-03-05 19:54:00,548 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for aa3a9f808cf1db1e0667a9fff093c3ec: 2023-03-05 19:54:00,548 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1678046035983.aa3a9f808cf1db1e0667a9fff093c3ec. 2023-03-05 19:54:00,549 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-05 19:54:00,549 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-05 19:54:00,549 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:00,549 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-05 19:54:00,549 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-05 19:54:00,550 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:00,550 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:00,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:00,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:00,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:00,551 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:00,551 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-05 19:54:00,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:00,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:00,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:00,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:00,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:00,552 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-05 19:54:00,552 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4106620c[Count = 0] remaining members to acquire global barrier 2023-03-05 19:54:00,552 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-05 19:54:00,552 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:00,554 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:00,554 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:00,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:00,554 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-05 19:54:00,554 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-05 19:54:00,554 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,37037,1678046018983' in zk 2023-03-05 19:54:00,554 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:00,554 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-05 19:54:00,556 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-05 19:54:00,556 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:00,556 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:54:00,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:00,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:00,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:00,556 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-05 19:54:00,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:00,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:00,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:00,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:00,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:00,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:00,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:00,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,37037,1678046018983': 2023-03-05 19:54:00,560 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-05 19:54:00,560 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-05 19:54:00,560 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-05 19:54:00,560 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-05 19:54:00,560 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:00,560 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-05 19:54:00,562 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:00,562 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:00,562 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:00,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:00,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:00,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:00,562 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:54:00,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:00,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:00,563 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:00,563 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:54:00,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:54:00,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:00,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:00,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:00,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:00,564 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:00,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:00,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:00,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:00,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:00,567 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:00,567 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:00,567 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:00,567 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:00,567 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:00,567 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-05 19:54:00,567 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-05 19:54:00,567 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:00,567 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:54:00,567 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:54:00,567 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:00,568 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:54:00,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:54:00,567 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:54:00,568 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-05 19:54:00,568 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-05 19:54:00,568 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:54:00,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:54:00,668 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-05 19:54:00,670 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-05 19:54:00,674 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-05 19:54:00,676 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-05 19:54:00,676 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:00,676 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-05 19:54:00,676 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:54:00,677 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-05 19:54:00,677 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-05 19:54:00,677 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:00,678 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:00,679 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:54:00,679 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:00,679 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:54:00,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:54:00,679 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:00,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:00,679 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-05 19:54:00,680 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:00,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-05 19:54:00,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:00,680 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:00,680 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-03-05 19:54:00,680 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:00,680 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-05 19:54:00,680 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:54:00,681 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-05 19:54:00,681 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-05 19:54:00,681 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-05 19:54:00,681 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1678046035983.aa3a9f808cf1db1e0667a9fff093c3ec. 2023-03-05 19:54:00,681 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1678046035983.aa3a9f808cf1db1e0667a9fff093c3ec. started... 2023-03-05 19:54:00,681 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing aa3a9f808cf1db1e0667a9fff093c3ec 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-05 19:54:01,097 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512055f776464d00e46239fe1a8db4dbf1521 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201512055f776464d00e46239fe1a8db4dbf1521 2023-03-05 19:54:01,506 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec/.tmp/family1/30d3b93f00eb4a8a81d454be03a1df84 2023-03-05 19:54:01,512 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec/.tmp/family1/30d3b93f00eb4a8a81d454be03a1df84 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec/family1/30d3b93f00eb4a8a81d454be03a1df84 2023-03-05 19:54:01,518 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec/family1/30d3b93f00eb4a8a81d454be03a1df84, entries=1, sequenceid=17, filesize=4.9 K 2023-03-05 19:54:01,519 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for aa3a9f808cf1db1e0667a9fff093c3ec in 838ms, sequenceid=17, compaction requested=true 2023-03-05 19:54:01,519 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for aa3a9f808cf1db1e0667a9fff093c3ec: 2023-03-05 19:54:01,519 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1678046035983.aa3a9f808cf1db1e0667a9fff093c3ec. 2023-03-05 19:54:01,519 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-05 19:54:01,519 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-05 19:54:01,519 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:01,519 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-05 19:54:01,519 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-05 19:54:01,522 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:01,522 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:01,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:01,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:01,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:01,523 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:01,523 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-05 19:54:01,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:01,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:01,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:01,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:01,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:01,524 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-05 19:54:01,524 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@59ee3b3f[Count = 0] remaining members to acquire global barrier 2023-03-05 19:54:01,524 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-05 19:54:01,524 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:01,525 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:01,526 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:01,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:01,526 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-05 19:54:01,526 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-05 19:54:01,526 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,37037,1678046018983' in zk 2023-03-05 19:54:01,526 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:01,526 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-05 19:54:01,527 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-05 19:54:01,527 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:01,527 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:54:01,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:01,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:01,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:01,527 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-05 19:54:01,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:01,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:01,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:01,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:01,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:01,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:01,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:01,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,37037,1678046018983': 2023-03-05 19:54:01,531 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-05 19:54:01,531 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-05 19:54:01,531 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-05 19:54:01,531 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-05 19:54:01,531 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:01,531 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-05 19:54:01,533 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:01,533 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:01,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:01,533 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:01,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:01,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:01,533 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:54:01,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:01,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:01,533 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:01,533 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:54:01,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:54:01,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:01,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:01,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:01,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:01,535 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:01,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:01,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:01,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:01,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:01,543 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:01,543 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:54:01,544 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:54:01,544 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:01,544 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:54:01,544 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-05 19:54:01,544 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-05 19:54:01,544 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:54:01,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:54:01,544 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:01,545 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:01,545 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:01,545 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:01,545 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:01,545 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-05 19:54:01,545 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-05 19:54:01,545 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:54:01,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:54:01,645 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-05 19:54:01,646 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-05 19:54:01,651 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-05 19:54:01,653 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-05 19:54:01,653 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:01,653 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-05 19:54:01,653 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:54:01,654 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-05 19:54:01,654 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-05 19:54:01,654 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:01,654 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:01,656 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:01,656 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:54:01,656 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:54:01,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:54:01,656 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:01,656 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-05 19:54:01,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:01,657 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:01,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-05 19:54:01,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:01,657 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:01,657 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-03-05 19:54:01,657 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:01,657 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-05 19:54:01,657 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:54:01,658 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-05 19:54:01,658 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-05 19:54:01,658 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-05 19:54:01,658 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1678046035983.aa3a9f808cf1db1e0667a9fff093c3ec. 2023-03-05 19:54:01,658 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1678046035983.aa3a9f808cf1db1e0667a9fff093c3ec. started... 2023-03-05 19:54:01,658 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing aa3a9f808cf1db1e0667a9fff093c3ec 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-05 19:54:01,675 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015122861ff3637204f497588d73b17805b331a to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015122861ff3637204f497588d73b17805b331a 2023-03-05 19:54:02,081 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec/.tmp/family1/94b8a0ea81cc4ec89da235a0f6d52bb3 2023-03-05 19:54:02,087 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec/.tmp/family1/94b8a0ea81cc4ec89da235a0f6d52bb3 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec/family1/94b8a0ea81cc4ec89da235a0f6d52bb3 2023-03-05 19:54:02,094 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec/family1/94b8a0ea81cc4ec89da235a0f6d52bb3, entries=1, sequenceid=21, filesize=4.9 K 2023-03-05 19:54:02,095 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for aa3a9f808cf1db1e0667a9fff093c3ec in 437ms, sequenceid=21, compaction requested=true 2023-03-05 19:54:02,095 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for aa3a9f808cf1db1e0667a9fff093c3ec: 2023-03-05 19:54:02,095 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1678046035983.aa3a9f808cf1db1e0667a9fff093c3ec. 2023-03-05 19:54:02,095 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-05 19:54:02,095 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-05 19:54:02,095 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:02,095 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-05 19:54:02,095 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-05 19:54:02,097 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:02,097 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:02,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:02,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:02,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:02,097 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:02,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:02,097 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-05 19:54:02,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:02,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:02,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:02,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:02,099 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-05 19:54:02,099 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6f9179e6[Count = 0] remaining members to acquire global barrier 2023-03-05 19:54:02,099 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-05 19:54:02,099 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:02,100 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:02,100 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:02,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:02,101 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-05 19:54:02,101 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-05 19:54:02,101 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,37037,1678046018983' in zk 2023-03-05 19:54:02,101 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:02,101 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-05 19:54:02,103 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-05 19:54:02,103 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:54:02,103 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:02,103 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-05 19:54:02,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:02,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:02,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:02,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:02,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:02,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:02,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:02,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:02,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:02,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:02,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,37037,1678046018983': 2023-03-05 19:54:02,107 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-05 19:54:02,107 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-05 19:54:02,107 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-05 19:54:02,107 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-05 19:54:02,107 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:02,107 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-05 19:54:02,111 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:02,111 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:02,111 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:02,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:02,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:02,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:02,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:02,111 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:54:02,112 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:54:02,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:02,112 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:02,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:54:02,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:02,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:02,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:02,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:02,113 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:02,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:02,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:02,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:02,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:02,116 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:54:02,116 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:54:02,116 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:54:02,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:54:02,116 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:54:02,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:54:02,116 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:02,116 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:02,116 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:02,116 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:02,116 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-05 19:54:02,116 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-05 19:54:02,116 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:54:02,116 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:02,117 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:02,117 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:02,117 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-05 19:54:02,118 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-05 19:54:02,218 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-05 19:54:02,219 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-05 19:54:02,224 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-05 19:54:02,225 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-05 19:54:02,226 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:02,226 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-05 19:54:02,226 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:54:02,226 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-05 19:54:02,226 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-05 19:54:02,227 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:02,227 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:02,229 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:54:02,230 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:02,230 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:54:02,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:54:02,230 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:02,230 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-05 19:54:02,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:02,230 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:02,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-05 19:54:02,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:02,231 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:02,232 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-03-05 19:54:02,232 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:02,232 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-05 19:54:02,232 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:54:02,233 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-05 19:54:02,233 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-05 19:54:02,233 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-05 19:54:02,233 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1678046035983.aa3a9f808cf1db1e0667a9fff093c3ec. 2023-03-05 19:54:02,233 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1678046035983.aa3a9f808cf1db1e0667a9fff093c3ec. started... 2023-03-05 19:54:02,233 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing aa3a9f808cf1db1e0667a9fff093c3ec 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-05 19:54:02,656 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015123130eadbaf869c420eb1d32e987817aee6 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015123130eadbaf869c420eb1d32e987817aee6 2023-03-05 19:54:03,063 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec/.tmp/family1/d5ac637c6e044495991db42ca7ccfec8 2023-03-05 19:54:03,070 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec/.tmp/family1/d5ac637c6e044495991db42ca7ccfec8 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec/family1/d5ac637c6e044495991db42ca7ccfec8 2023-03-05 19:54:03,075 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec/family1/d5ac637c6e044495991db42ca7ccfec8, entries=1, sequenceid=25, filesize=4.9 K 2023-03-05 19:54:03,077 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for aa3a9f808cf1db1e0667a9fff093c3ec in 844ms, sequenceid=25, compaction requested=true 2023-03-05 19:54:03,077 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for aa3a9f808cf1db1e0667a9fff093c3ec: 2023-03-05 19:54:03,077 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1678046035983.aa3a9f808cf1db1e0667a9fff093c3ec. 2023-03-05 19:54:03,077 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-05 19:54:03,077 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-05 19:54:03,077 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:03,077 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-05 19:54:03,077 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-05 19:54:03,079 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:03,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:03,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:03,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:03,080 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:03,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:03,080 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:03,080 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-05 19:54:03,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:03,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:03,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:03,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:03,082 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-05 19:54:03,082 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4ec31da9[Count = 0] remaining members to acquire global barrier 2023-03-05 19:54:03,082 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-05 19:54:03,082 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:03,083 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:03,084 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:03,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:03,084 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-05 19:54:03,084 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-05 19:54:03,084 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,37037,1678046018983' in zk 2023-03-05 19:54:03,084 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:03,084 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-05 19:54:03,085 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-05 19:54:03,085 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:03,085 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:54:03,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:03,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:03,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:03,086 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-05 19:54:03,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:03,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:03,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:03,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:03,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:03,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:03,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:03,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,37037,1678046018983': 2023-03-05 19:54:03,089 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-05 19:54:03,089 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-05 19:54:03,090 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-05 19:54:03,090 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-05 19:54:03,090 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:03,090 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-05 19:54:03,093 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:03,093 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:03,093 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:03,093 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:54:03,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:03,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:03,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:03,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:03,093 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:54:03,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:54:03,094 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:03,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:03,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:03,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:03,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:03,095 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:03,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:03,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:03,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:03,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:03,098 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:03,098 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:54:03,098 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:03,098 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:54:03,098 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:03,098 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:03,098 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:54:03,098 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:54:03,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:54:03,098 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:03,098 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-05 19:54:03,099 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-05 19:54:03,099 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:03,099 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:03,099 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:54:03,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:54:03,100 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-05 19:54:03,100 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-05 19:54:03,100 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-05 19:54:03,200 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-05 19:54:03,201 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-05 19:54:03,208 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-05 19:54:03,210 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-05 19:54:03,210 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:03,210 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-05 19:54:03,210 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:54:03,211 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-05 19:54:03,211 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-05 19:54:03,211 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:03,211 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:03,213 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:54:03,213 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:03,213 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:54:03,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:54:03,214 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:03,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:03,214 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-05 19:54:03,214 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:03,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-05 19:54:03,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:03,215 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:03,215 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-03-05 19:54:03,215 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:03,215 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-05 19:54:03,215 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:54:03,216 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-05 19:54:03,217 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-05 19:54:03,217 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-05 19:54:03,217 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1678046035983.aa3a9f808cf1db1e0667a9fff093c3ec. 2023-03-05 19:54:03,217 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1678046035983.aa3a9f808cf1db1e0667a9fff093c3ec. started... 2023-03-05 19:54:03,217 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing aa3a9f808cf1db1e0667a9fff093c3ec 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-05 19:54:03,641 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160101eca9d782bf8747588878811b20781f3b to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20160101eca9d782bf8747588878811b20781f3b 2023-03-05 19:54:04,047 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec/.tmp/family1/51997668919541c6a5a6290ad34ecf4d 2023-03-05 19:54:04,053 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec/.tmp/family1/51997668919541c6a5a6290ad34ecf4d as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec/family1/51997668919541c6a5a6290ad34ecf4d 2023-03-05 19:54:04,058 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec/family1/51997668919541c6a5a6290ad34ecf4d, entries=1, sequenceid=29, filesize=4.9 K 2023-03-05 19:54:04,059 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for aa3a9f808cf1db1e0667a9fff093c3ec in 842ms, sequenceid=29, compaction requested=true 2023-03-05 19:54:04,059 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for aa3a9f808cf1db1e0667a9fff093c3ec: 2023-03-05 19:54:04,059 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1678046035983.aa3a9f808cf1db1e0667a9fff093c3ec. 2023-03-05 19:54:04,059 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-05 19:54:04,059 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-05 19:54:04,059 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:04,059 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-05 19:54:04,059 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-05 19:54:04,061 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:04,061 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:04,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:04,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:04,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:04,061 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:04,061 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-05 19:54:04,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:04,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:04,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:04,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:04,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:04,062 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-05 19:54:04,063 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3ca7107f[Count = 0] remaining members to acquire global barrier 2023-03-05 19:54:04,063 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-05 19:54:04,063 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:04,064 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:04,064 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:04,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:04,064 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-05 19:54:04,064 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-05 19:54:04,064 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:04,064 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-05 19:54:04,064 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,37037,1678046018983' in zk 2023-03-05 19:54:04,066 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:04,066 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-05 19:54:04,066 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:54:04,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:04,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:04,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:04,066 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-05 19:54:04,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:04,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:04,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:04,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:04,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:04,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:04,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:04,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,37037,1678046018983': 2023-03-05 19:54:04,069 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-05 19:54:04,069 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-05 19:54:04,069 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-05 19:54:04,069 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-05 19:54:04,069 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:04,069 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-05 19:54:04,072 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:04,072 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:04,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:04,072 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:04,072 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:54:04,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:04,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:04,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:04,073 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:54:04,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:54:04,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:04,073 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:04,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:04,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:04,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:04,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:04,074 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:04,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:04,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:04,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:04,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:04,076 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:04,076 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:54:04,076 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:04,076 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-05 19:54:04,076 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:54:04,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:54:04,076 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:54:04,076 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:04,077 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:04,077 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:04,077 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:04,077 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:04,077 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:54:04,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:54:04,077 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-05 19:54:04,077 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-05 19:54:04,077 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:54:04,078 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-05 19:54:04,078 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-05 19:54:04,178 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-05 19:54:04,179 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-05 19:54:04,185 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-05 19:54:04,187 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-05 19:54:04,187 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:04,187 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-05 19:54:04,187 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:54:04,188 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-05 19:54:04,188 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-05 19:54:04,188 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:04,188 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:04,189 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:54:04,189 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:04,189 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:54:04,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:54:04,190 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:04,190 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-05 19:54:04,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:04,190 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:04,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-05 19:54:04,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:04,191 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:04,191 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-03-05 19:54:04,191 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:04,191 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-05 19:54:04,191 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:54:04,191 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-05 19:54:04,192 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-05 19:54:04,192 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-05 19:54:04,192 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1678046035983.aa3a9f808cf1db1e0667a9fff093c3ec. 2023-03-05 19:54:04,192 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1678046035983.aa3a9f808cf1db1e0667a9fff093c3ec. started... 2023-03-05 19:54:04,192 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing aa3a9f808cf1db1e0667a9fff093c3ec 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-05 19:54:04,606 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2016010354c2d94df37c42cd822f15f1358e3885 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2016010354c2d94df37c42cd822f15f1358e3885 2023-03-05 19:54:05,013 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec/.tmp/family1/81346b9b87114dd1aeb1b11eaff51592 2023-03-05 19:54:05,019 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec/.tmp/family1/81346b9b87114dd1aeb1b11eaff51592 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec/family1/81346b9b87114dd1aeb1b11eaff51592 2023-03-05 19:54:05,024 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec/family1/81346b9b87114dd1aeb1b11eaff51592, entries=1, sequenceid=33, filesize=4.9 K 2023-03-05 19:54:05,025 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for aa3a9f808cf1db1e0667a9fff093c3ec in 833ms, sequenceid=33, compaction requested=true 2023-03-05 19:54:05,025 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for aa3a9f808cf1db1e0667a9fff093c3ec: 2023-03-05 19:54:05,025 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1678046035983.aa3a9f808cf1db1e0667a9fff093c3ec. 2023-03-05 19:54:05,025 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-05 19:54:05,025 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-05 19:54:05,025 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:05,025 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-05 19:54:05,025 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-05 19:54:05,027 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:05,027 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:05,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:05,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:05,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:05,027 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:05,027 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-05 19:54:05,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:05,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:05,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:05,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:05,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:05,029 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-05 19:54:05,029 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1438f317[Count = 0] remaining members to acquire global barrier 2023-03-05 19:54:05,029 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-05 19:54:05,029 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:05,034 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:05,034 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:05,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:05,034 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-05 19:54:05,034 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:05,034 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-05 19:54:05,034 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-05 19:54:05,034 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,37037,1678046018983' in zk 2023-03-05 19:54:05,044 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-05 19:54:05,044 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:05,044 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:54:05,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:05,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:05,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:05,044 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-05 19:54:05,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:05,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:05,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:05,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:05,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:05,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:05,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:05,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,37037,1678046018983': 2023-03-05 19:54:05,046 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-05 19:54:05,046 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-05 19:54:05,047 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-05 19:54:05,047 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-05 19:54:05,047 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:05,047 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-05 19:54:05,048 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:05,048 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:05,048 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:05,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:05,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:05,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:05,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:05,048 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:54:05,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:05,049 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:54:05,049 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:05,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:54:05,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:05,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:05,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:05,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:05,050 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:05,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:05,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:05,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:05,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:05,052 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:05,052 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:54:05,052 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:54:05,052 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:05,052 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:54:05,052 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-05 19:54:05,053 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-05 19:54:05,052 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:54:05,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:54:05,053 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:05,053 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:05,053 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:05,053 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:05,053 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:05,053 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-05 19:54:05,053 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-05 19:54:05,053 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:54:05,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:54:05,153 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-05 19:54:05,155 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-05 19:54:05,159 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-05 19:54:05,161 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-05 19:54:05,161 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:05,162 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-05 19:54:05,162 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:54:05,163 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-05 19:54:05,163 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-05 19:54:05,163 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:05,163 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:05,166 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:54:05,166 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:05,166 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:54:05,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:54:05,166 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:05,167 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-05 19:54:05,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:05,167 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:05,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-05 19:54:05,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:05,167 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:05,168 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-03-05 19:54:05,168 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:05,168 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-05 19:54:05,168 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:54:05,168 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-05 19:54:05,168 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-05 19:54:05,168 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-05 19:54:05,168 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1678046035983.aa3a9f808cf1db1e0667a9fff093c3ec. 2023-03-05 19:54:05,169 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1678046035983.aa3a9f808cf1db1e0667a9fff093c3ec. started... 2023-03-05 19:54:05,169 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing aa3a9f808cf1db1e0667a9fff093c3ec 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-05 19:54:05,188 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20150907ca13a3304d49419783fc614e4f58c422 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20150907ca13a3304d49419783fc614e4f58c422 2023-03-05 19:54:05,597 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec/.tmp/family1/2bab25a112aa456cb6e979740b4b82ec 2023-03-05 19:54:05,603 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec/.tmp/family1/2bab25a112aa456cb6e979740b4b82ec as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec/family1/2bab25a112aa456cb6e979740b4b82ec 2023-03-05 19:54:05,607 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec/family1/2bab25a112aa456cb6e979740b4b82ec, entries=1, sequenceid=37, filesize=4.9 K 2023-03-05 19:54:05,609 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for aa3a9f808cf1db1e0667a9fff093c3ec in 440ms, sequenceid=37, compaction requested=true 2023-03-05 19:54:05,609 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for aa3a9f808cf1db1e0667a9fff093c3ec: 2023-03-05 19:54:05,609 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1678046035983.aa3a9f808cf1db1e0667a9fff093c3ec. 2023-03-05 19:54:05,609 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-05 19:54:05,609 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-05 19:54:05,609 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:05,609 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-05 19:54:05,609 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-05 19:54:05,611 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:05,611 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:05,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:05,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:05,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:05,611 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:05,611 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-05 19:54:05,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:05,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:05,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:05,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:05,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:05,613 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-05 19:54:05,613 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7082e63f[Count = 0] remaining members to acquire global barrier 2023-03-05 19:54:05,613 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-05 19:54:05,613 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:05,614 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:05,614 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:05,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:05,614 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-05 19:54:05,614 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:05,614 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-05 19:54:05,614 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-05 19:54:05,614 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,37037,1678046018983' in zk 2023-03-05 19:54:05,616 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-05 19:54:05,616 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:05,616 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:54:05,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:05,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:05,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:05,616 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-05 19:54:05,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:05,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:05,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:05,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:05,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:05,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:05,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:05,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,37037,1678046018983': 2023-03-05 19:54:05,619 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-05 19:54:05,619 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-05 19:54:05,619 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-05 19:54:05,619 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-05 19:54:05,619 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:05,619 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-05 19:54:05,622 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:05,622 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:05,622 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:05,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:05,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:05,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:05,622 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:54:05,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:05,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:05,622 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:05,622 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:54:05,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:54:05,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:05,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:05,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:05,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:05,624 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:05,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:05,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:05,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:05,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:05,626 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:05,626 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:05,626 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:54:05,626 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:54:05,626 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:54:05,626 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:05,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:54:05,626 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-05 19:54:05,626 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-05 19:54:05,627 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-05 19:54:05,626 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:54:05,627 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:05,627 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:05,627 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:05,627 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:05,627 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:54:05,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:54:05,627 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-05 19:54:05,627 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-05 19:54:05,727 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-05 19:54:05,728 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-05 19:54:05,733 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-05 19:54:05,735 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-05 19:54:05,735 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:05,735 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-05 19:54:05,735 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:54:05,736 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-05 19:54:05,736 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-05 19:54:05,736 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:05,736 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:05,737 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:05,737 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:54:05,738 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:54:05,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:54:05,738 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:05,738 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-05 19:54:05,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:05,738 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:05,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-05 19:54:05,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:05,739 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:05,739 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-03-05 19:54:05,739 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:05,739 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-05 19:54:05,739 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:54:05,740 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-05 19:54:05,740 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-05 19:54:05,740 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-05 19:54:05,740 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1678046035983.aa3a9f808cf1db1e0667a9fff093c3ec. 2023-03-05 19:54:05,741 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1678046035983.aa3a9f808cf1db1e0667a9fff093c3ec. started... 2023-03-05 19:54:05,741 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing aa3a9f808cf1db1e0667a9fff093c3ec 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-05 19:54:06,155 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151120a8bbe18d5d794fc6be68a64ac39fbb4b to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151120a8bbe18d5d794fc6be68a64ac39fbb4b 2023-03-05 19:54:06,561 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec/.tmp/family1/0653ca4b03ca4e958006df84a15dcc57 2023-03-05 19:54:06,567 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec/.tmp/family1/0653ca4b03ca4e958006df84a15dcc57 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec/family1/0653ca4b03ca4e958006df84a15dcc57 2023-03-05 19:54:06,571 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec/family1/0653ca4b03ca4e958006df84a15dcc57, entries=1, sequenceid=41, filesize=4.9 K 2023-03-05 19:54:06,572 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for aa3a9f808cf1db1e0667a9fff093c3ec in 831ms, sequenceid=41, compaction requested=true 2023-03-05 19:54:06,572 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for aa3a9f808cf1db1e0667a9fff093c3ec: 2023-03-05 19:54:06,572 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1678046035983.aa3a9f808cf1db1e0667a9fff093c3ec. 2023-03-05 19:54:06,573 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-05 19:54:06,573 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-05 19:54:06,573 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:06,573 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-05 19:54:06,573 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-05 19:54:06,574 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:06,574 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:06,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:06,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:06,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:06,575 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:06,575 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-05 19:54:06,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:06,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:06,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:06,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:06,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:06,577 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-05 19:54:06,577 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@267a83d3[Count = 0] remaining members to acquire global barrier 2023-03-05 19:54:06,577 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-05 19:54:06,577 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:06,579 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:06,579 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:06,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:06,579 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-05 19:54:06,579 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-05 19:54:06,579 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:06,579 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-05 19:54:06,579 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,37037,1678046018983' in zk 2023-03-05 19:54:06,580 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:06,580 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-05 19:54:06,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:06,580 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:54:06,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:06,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:06,581 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-05 19:54:06,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:06,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:06,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:06,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:06,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:06,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:06,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:06,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,37037,1678046018983': 2023-03-05 19:54:06,583 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-05 19:54:06,583 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-05 19:54:06,583 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-05 19:54:06,583 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-05 19:54:06,583 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:06,583 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-05 19:54:06,584 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:06,584 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:06,585 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:06,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:06,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:06,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:06,585 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:54:06,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:06,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:06,585 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:06,585 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:54:06,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:54:06,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:06,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:06,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:06,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:06,586 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:06,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:06,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:06,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:06,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:06,589 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:06,589 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:54:06,589 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:54:06,589 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:06,589 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-05 19:54:06,590 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-05 19:54:06,589 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:54:06,590 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-05 19:54:06,589 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:54:06,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:54:06,590 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:06,590 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-05 19:54:06,590 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:06,590 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-05 19:54:06,590 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:54:06,590 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:06,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:54:06,591 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:06,591 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:06,691 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-05 19:54:06,692 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-05 19:54:06,693 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=38221] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMajorCompactionWithMonthlyPolicy 2023-03-05 19:54:06,695 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] procedure2.ProcedureExecutor(1029): Stored pid=22, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-03-05 19:54:06,695 DEBUG [PEWorker-5] locking.LockProcedure(309): LOCKED pid=22, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-03-05 19:54:06,696 INFO [PEWorker-5] procedure2.TimeoutExecutorThread(81): ADDED pid=22, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678046646696 2023-03-05 19:54:06,696 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithMonthlyPolicy', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-05 19:54:06,698 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-05 19:54:06,699 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithMonthlyPolicy' and column 'family1' 2023-03-05 19:54:06,699 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-03-05 19:54:06,699 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] zookeeper.ReadOnlyZKClient(139): Connect 0x564e1028 to 127.0.0.1:62957 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-05 19:54:06,705 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@49bbe55d, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-05 19:54:06,705 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-03-05 19:54:07,543 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103973ecacdffe8440ba0ee43ace8cd9495 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20160103973ecacdffe8440ba0ee43ace8cd9495 2023-03-05 19:54:07,544 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20160101 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-03-05 19:54:07,557 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-05 19:54:07,558 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:38430, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-05 19:54:07,565 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-05 19:54:07,566 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:52822, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-05 19:54:07,576 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/6c4d7bca10e84c94a5eb865d6d73f461 first=Optional[r5] last=Optional[r6] 2023-03-05 19:54:07,576 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithMonthlyPolicy,,1678046035983.aa3a9f808cf1db1e0667a9fff093c3ec., hostname=jenkins-hbase4.apache.org,37037,1678046018983, seqNum=2 for row with hfile group [{family1,hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/6c4d7bca10e84c94a5eb865d6d73f461}] 2023-03-05 19:54:07,582 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(763): Validating hfile at hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/6c4d7bca10e84c94a5eb865d6d73f461 for inclusion in aa3a9f808cf1db1e0667a9fff093c3ec/family1 2023-03-05 19:54:07,585 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(775): HFile bounds: first=r5 last=r6 2023-03-05 19:54:07,585 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(777): Region bounds: first= last= 2023-03-05 19:54:07,585 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HRegion(2446): Flush status journal for aa3a9f808cf1db1e0667a9fff093c3ec: 2023-03-05 19:54:07,586 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/6c4d7bca10e84c94a5eb865d6d73f461 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testMajorCompactionWithMonthlyPolicy__ic1pb49lrrb5r46cptqj2krbeuto2panh5ipor4vfprm4nnnmphb0v9qq96n4a57/family1/6c4d7bca10e84c94a5eb865d6d73f461 2023-03-05 19:54:07,588 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testMajorCompactionWithMonthlyPolicy__ic1pb49lrrb5r46cptqj2krbeuto2panh5ipor4vfprm4nnnmphb0v9qq96n4a57/family1/6c4d7bca10e84c94a5eb865d6d73f461 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec/family1/1798aefe5011446d9a965d5ee747cd23_SeqId_44_ 2023-03-05 19:54:07,589 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(849): Loaded HFile hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testMajorCompactionWithMonthlyPolicy__ic1pb49lrrb5r46cptqj2krbeuto2panh5ipor4vfprm4nnnmphb0v9qq96n4a57/family1/6c4d7bca10e84c94a5eb865d6d73f461 into aa3a9f808cf1db1e0667a9fff093c3ec/family1 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec/family1/1798aefe5011446d9a965d5ee747cd23_SeqId_44_ - updating store file list. 2023-03-05 19:54:07,595 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(882): Loaded HFile hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec/family1/1798aefe5011446d9a965d5ee747cd23_SeqId_44_ into aa3a9f808cf1db1e0667a9fff093c3ec/family1 2023-03-05 19:54:07,595 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(855): Successfully loaded hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testMajorCompactionWithMonthlyPolicy__ic1pb49lrrb5r46cptqj2krbeuto2panh5ipor4vfprm4nnnmphb0v9qq96n4a57/family1/6c4d7bca10e84c94a5eb865d6d73f461 into aa3a9f808cf1db1e0667a9fff093c3ec/family1 (new location: hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec/family1/1798aefe5011446d9a965d5ee747cd23_SeqId_44_) 2023-03-05 19:54:07,596 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testMajorCompactionWithMonthlyPolicy__ic1pb49lrrb5r46cptqj2krbeuto2panh5ipor4vfprm4nnnmphb0v9qq96n4a57/family1/6c4d7bca10e84c94a5eb865d6d73f461 2023-03-05 19:54:07,596 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-03-05 19:54:07,599 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 50903 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-03-05 19:54:07,600 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(1912): aa3a9f808cf1db1e0667a9fff093c3ec/family1 is initiating minor compaction 2023-03-05 19:54:07,600 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@45661892; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-05 19:54:07,600 INFO [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of aa3a9f808cf1db1e0667a9fff093c3ec/family1 in testMajorCompactionWithMonthlyPolicy,,1678046035983.aa3a9f808cf1db1e0667a9fff093c3ec. 2023-03-05 19:54:07,600 INFO [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec/family1/194ecb5ba7cd44fa9349d80c8ca881fe, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec/family1/d2a282c260cb49e5b461a2e98c567b19, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec/family1/30d3b93f00eb4a8a81d454be03a1df84, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec/family1/94b8a0ea81cc4ec89da235a0f6d52bb3, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec/family1/d5ac637c6e044495991db42ca7ccfec8, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec/family1/51997668919541c6a5a6290ad34ecf4d, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec/family1/81346b9b87114dd1aeb1b11eaff51592, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec/family1/2bab25a112aa456cb6e979740b4b82ec, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec/family1/0653ca4b03ca4e958006df84a15dcc57, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec/family1/1798aefe5011446d9a965d5ee747cd23_SeqId_44_] into tmpdir=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec/.tmp, totalSize=49.7 K 2023-03-05 19:54:07,600 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.Compactor(207): Compacting 194ecb5ba7cd44fa9349d80c8ca881fe, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-03-05 19:54:07,600 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.Compactor(207): Compacting d2a282c260cb49e5b461a2e98c567b19, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=13 2023-03-05 19:54:07,600 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.Compactor(207): Compacting 30d3b93f00eb4a8a81d454be03a1df84, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=17 2023-03-05 19:54:07,600 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.Compactor(207): Compacting 94b8a0ea81cc4ec89da235a0f6d52bb3, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=21 2023-03-05 19:54:07,600 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.Compactor(207): Compacting d5ac637c6e044495991db42ca7ccfec8, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=25 2023-03-05 19:54:07,600 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.Compactor(207): Compacting 51997668919541c6a5a6290ad34ecf4d, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=29 2023-03-05 19:54:07,601 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.Compactor(207): Compacting 81346b9b87114dd1aeb1b11eaff51592, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=33 2023-03-05 19:54:07,601 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.Compactor(207): Compacting 2bab25a112aa456cb6e979740b4b82ec, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=37 2023-03-05 19:54:07,601 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.Compactor(207): Compacting 0653ca4b03ca4e958006df84a15dcc57, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=41 2023-03-05 19:54:07,601 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.Compactor(207): Compacting 1798aefe5011446d9a965d5ee747cd23_SeqId_44_, keycount=2, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=45 2023-03-05 19:54:07,602 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20160101 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-03-05 19:54:07,603 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-05 19:54:07,605 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20160101eca9d782bf8747588878811b20781f3b to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20160101eca9d782bf8747588878811b20781f3b 2023-03-05 19:54:07,606 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2016010354c2d94df37c42cd822f15f1358e3885 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2016010354c2d94df37c42cd822f15f1358e3885 2023-03-05 19:54:07,606 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithMonthlyPolicy 2 expired mob files are deleted 2023-03-05 19:54:07,606 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101' 2023-03-05 19:54:07,607 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-03-05 19:54:07,607 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-03-05 19:54:07,641 INFO [RS:0;jenkins-hbase4:37037-shortCompactions-0] throttle.PressureAwareThroughputController(145): aa3a9f808cf1db1e0667a9fff093c3ec#family1#compaction#22 average throughput is 1.37 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-05 19:54:08,457 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec/.tmp/family1/293e83fcfcac495d88a4f698553f6da5 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec/family1/293e83fcfcac495d88a4f698553f6da5 2023-03-05 19:54:08,460 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231d62e120a42de41d8a860f637fd19ce8e to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151231d62e120a42de41d8a860f637fd19ce8e 2023-03-05 19:54:08,461 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151201 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-03-05 19:54:08,463 INFO [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in aa3a9f808cf1db1e0667a9fff093c3ec/family1 of aa3a9f808cf1db1e0667a9fff093c3ec into 293e83fcfcac495d88a4f698553f6da5(size=6.6 K), total size for store is 172.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-05 19:54:08,463 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for aa3a9f808cf1db1e0667a9fff093c3ec: 2023-03-05 19:54:08,463 INFO [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithMonthlyPolicy,,1678046035983.aa3a9f808cf1db1e0667a9fff093c3ec., storeName=family1, fileCount=10, fileSize=49.7 K (5.0 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 5.2 K), priority=2, time=1678046047600; duration=0sec 2023-03-05 19:54:08,464 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-05 19:54:08,485 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/b97b8f46de684eb7bfb9f79082ca0540 first=Optional[r1] last=Optional[r4] 2023-03-05 19:54:08,485 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithMonthlyPolicy,,1678046035983.aa3a9f808cf1db1e0667a9fff093c3ec., hostname=jenkins-hbase4.apache.org,37037,1678046018983, seqNum=2 for row with hfile group [{family1,hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/b97b8f46de684eb7bfb9f79082ca0540}] 2023-03-05 19:54:08,487 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(763): Validating hfile at hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/b97b8f46de684eb7bfb9f79082ca0540 for inclusion in aa3a9f808cf1db1e0667a9fff093c3ec/family1 2023-03-05 19:54:08,490 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(775): HFile bounds: first=r1 last=r4 2023-03-05 19:54:08,490 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(777): Region bounds: first= last= 2023-03-05 19:54:08,490 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HRegion(2446): Flush status journal for aa3a9f808cf1db1e0667a9fff093c3ec: 2023-03-05 19:54:08,490 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/b97b8f46de684eb7bfb9f79082ca0540 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testMajorCompactionWithMonthlyPolicy__1fu347ai9204a1bpek7iv6hclt3c4gbacaeidqa33d2e9bkhsuu2skdi7hr4u9hc/family1/b97b8f46de684eb7bfb9f79082ca0540 2023-03-05 19:54:08,493 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testMajorCompactionWithMonthlyPolicy__1fu347ai9204a1bpek7iv6hclt3c4gbacaeidqa33d2e9bkhsuu2skdi7hr4u9hc/family1/b97b8f46de684eb7bfb9f79082ca0540 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec/family1/1fa6b060683a4225934b8332b06b92f6_SeqId_47_ 2023-03-05 19:54:08,494 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(849): Loaded HFile hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testMajorCompactionWithMonthlyPolicy__1fu347ai9204a1bpek7iv6hclt3c4gbacaeidqa33d2e9bkhsuu2skdi7hr4u9hc/family1/b97b8f46de684eb7bfb9f79082ca0540 into aa3a9f808cf1db1e0667a9fff093c3ec/family1 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec/family1/1fa6b060683a4225934b8332b06b92f6_SeqId_47_ - updating store file list. 2023-03-05 19:54:08,504 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec/family1/194ecb5ba7cd44fa9349d80c8ca881fe, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec/family1/d2a282c260cb49e5b461a2e98c567b19, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec/family1/30d3b93f00eb4a8a81d454be03a1df84, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec/family1/94b8a0ea81cc4ec89da235a0f6d52bb3, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec/family1/d5ac637c6e044495991db42ca7ccfec8, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec/family1/51997668919541c6a5a6290ad34ecf4d, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec/family1/81346b9b87114dd1aeb1b11eaff51592, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec/family1/2bab25a112aa456cb6e979740b4b82ec, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec/family1/0653ca4b03ca4e958006df84a15dcc57, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec/family1/1798aefe5011446d9a965d5ee747cd23_SeqId_44_] to archive 2023-03-05 19:54:08,504 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(882): Loaded HFile hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec/family1/1fa6b060683a4225934b8332b06b92f6_SeqId_47_ into aa3a9f808cf1db1e0667a9fff093c3ec/family1 2023-03-05 19:54:08,504 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(855): Successfully loaded hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testMajorCompactionWithMonthlyPolicy__1fu347ai9204a1bpek7iv6hclt3c4gbacaeidqa33d2e9bkhsuu2skdi7hr4u9hc/family1/b97b8f46de684eb7bfb9f79082ca0540 into aa3a9f808cf1db1e0667a9fff093c3ec/family1 (new location: hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec/family1/1fa6b060683a4225934b8332b06b92f6_SeqId_47_) 2023-03-05 19:54:08,505 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testMajorCompactionWithMonthlyPolicy__1fu347ai9204a1bpek7iv6hclt3c4gbacaeidqa33d2e9bkhsuu2skdi7hr4u9hc/family1/b97b8f46de684eb7bfb9f79082ca0540 2023-03-05 19:54:08,505 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-05 19:54:08,505 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-03-05 19:54:08,507 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 182349 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-03-05 19:54:08,507 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(1912): aa3a9f808cf1db1e0667a9fff093c3ec/family1 is initiating minor compaction (all files) 2023-03-05 19:54:08,507 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@6902187a; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-05 19:54:08,507 INFO [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of aa3a9f808cf1db1e0667a9fff093c3ec/family1 in testMajorCompactionWithMonthlyPolicy,,1678046035983.aa3a9f808cf1db1e0667a9fff093c3ec. 2023-03-05 19:54:08,507 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec/family1/194ecb5ba7cd44fa9349d80c8ca881fe to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec/family1/194ecb5ba7cd44fa9349d80c8ca881fe 2023-03-05 19:54:08,507 INFO [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec/family1/bc58a3df1dd0415fbe3b9606b4318973, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec/family1/293e83fcfcac495d88a4f698553f6da5, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec/family1/1fa6b060683a4225934b8332b06b92f6_SeqId_47_] into tmpdir=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec/.tmp, totalSize=178.1 K 2023-03-05 19:54:08,508 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.Compactor(207): Compacting bc58a3df1dd0415fbe3b9606b4318973, keycount=1000, bloomtype=ROW, size=166.0 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878843274 2023-03-05 19:54:08,509 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.Compactor(207): Compacting 293e83fcfcac495d88a4f698553f6da5, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621243274 2023-03-05 19:54:08,509 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec/family1/d2a282c260cb49e5b461a2e98c567b19 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec/family1/d2a282c260cb49e5b461a2e98c567b19 2023-03-05 19:54:08,510 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.Compactor(207): Compacting 1fa6b060683a4225934b8332b06b92f6_SeqId_47_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448965243274 2023-03-05 19:54:08,511 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec/family1/30d3b93f00eb4a8a81d454be03a1df84 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec/family1/30d3b93f00eb4a8a81d454be03a1df84 2023-03-05 19:54:08,513 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec/family1/94b8a0ea81cc4ec89da235a0f6d52bb3 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec/family1/94b8a0ea81cc4ec89da235a0f6d52bb3 2023-03-05 19:54:08,514 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec/family1/d5ac637c6e044495991db42ca7ccfec8 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec/family1/d5ac637c6e044495991db42ca7ccfec8 2023-03-05 19:54:08,516 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec/family1/51997668919541c6a5a6290ad34ecf4d to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec/family1/51997668919541c6a5a6290ad34ecf4d 2023-03-05 19:54:08,516 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151201 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-03-05 19:54:08,517 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-05 19:54:08,518 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec/family1/81346b9b87114dd1aeb1b11eaff51592 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec/family1/81346b9b87114dd1aeb1b11eaff51592 2023-03-05 19:54:08,519 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015120140465371c4bb4199aeab0076c6eba76f to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015120140465371c4bb4199aeab0076c6eba76f 2023-03-05 19:54:08,519 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec/family1/2bab25a112aa456cb6e979740b4b82ec to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec/family1/2bab25a112aa456cb6e979740b4b82ec 2023-03-05 19:54:08,520 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201512055f776464d00e46239fe1a8db4dbf1521 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201512055f776464d00e46239fe1a8db4dbf1521 2023-03-05 19:54:08,521 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec/family1/0653ca4b03ca4e958006df84a15dcc57 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec/family1/0653ca4b03ca4e958006df84a15dcc57 2023-03-05 19:54:08,522 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015122861ff3637204f497588d73b17805b331a to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015122861ff3637204f497588d73b17805b331a 2023-03-05 19:54:08,522 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec/family1/1798aefe5011446d9a965d5ee747cd23_SeqId_44_ to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec/family1/1798aefe5011446d9a965d5ee747cd23_SeqId_44_ 2023-03-05 19:54:08,523 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015123130eadbaf869c420eb1d32e987817aee6 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015123130eadbaf869c420eb1d32e987817aee6 2023-03-05 19:54:08,523 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithMonthlyPolicy 4 expired mob files are deleted 2023-03-05 19:54:08,523 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201' 2023-03-05 19:54:08,524 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-03-05 19:54:08,524 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-03-05 19:54:08,533 INFO [RS:0;jenkins-hbase4:37037-shortCompactions-0] throttle.PressureAwareThroughputController(145): aa3a9f808cf1db1e0667a9fff093c3ec#family1#compaction#23 average throughput is 12.97 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-05 19:54:08,585 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130fcca35c2f4ed4d65a5e4d724cd193f6f to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151130fcca35c2f4ed4d65a5e4d724cd193f6f 2023-03-05 19:54:08,586 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-03-05 19:54:08,617 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/6732698bc8c8474ba0e8e10ca0502483 first=Optional[r00] last=Optional[r8] 2023-03-05 19:54:08,618 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithMonthlyPolicy,,1678046035983.aa3a9f808cf1db1e0667a9fff093c3ec., hostname=jenkins-hbase4.apache.org,37037,1678046018983, seqNum=2 for row with hfile group [{family1,hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/6732698bc8c8474ba0e8e10ca0502483}] 2023-03-05 19:54:08,620 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(763): Validating hfile at hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/6732698bc8c8474ba0e8e10ca0502483 for inclusion in aa3a9f808cf1db1e0667a9fff093c3ec/family1 2023-03-05 19:54:08,623 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(775): HFile bounds: first=r00 last=r8 2023-03-05 19:54:08,623 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(777): Region bounds: first= last= 2023-03-05 19:54:08,623 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HRegion(2446): Flush status journal for aa3a9f808cf1db1e0667a9fff093c3ec: 2023-03-05 19:54:08,624 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/6732698bc8c8474ba0e8e10ca0502483 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testMajorCompactionWithMonthlyPolicy__5lrsvat5ipr0bl1djpail23b96s6pidl6s64bjtjvmqtpq8imrivinq8a63811sm/family1/6732698bc8c8474ba0e8e10ca0502483 2023-03-05 19:54:08,626 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testMajorCompactionWithMonthlyPolicy__5lrsvat5ipr0bl1djpail23b96s6pidl6s64bjtjvmqtpq8imrivinq8a63811sm/family1/6732698bc8c8474ba0e8e10ca0502483 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec/family1/c7f5beeb4749452c854751db0e71cf7e_SeqId_49_ 2023-03-05 19:54:08,626 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(849): Loaded HFile hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testMajorCompactionWithMonthlyPolicy__5lrsvat5ipr0bl1djpail23b96s6pidl6s64bjtjvmqtpq8imrivinq8a63811sm/family1/6732698bc8c8474ba0e8e10ca0502483 into aa3a9f808cf1db1e0667a9fff093c3ec/family1 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec/family1/c7f5beeb4749452c854751db0e71cf7e_SeqId_49_ - updating store file list. 2023-03-05 19:54:08,631 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(882): Loaded HFile hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec/family1/c7f5beeb4749452c854751db0e71cf7e_SeqId_49_ into aa3a9f808cf1db1e0667a9fff093c3ec/family1 2023-03-05 19:54:08,631 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(855): Successfully loaded hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testMajorCompactionWithMonthlyPolicy__5lrsvat5ipr0bl1djpail23b96s6pidl6s64bjtjvmqtpq8imrivinq8a63811sm/family1/6732698bc8c8474ba0e8e10ca0502483 into aa3a9f808cf1db1e0667a9fff093c3ec/family1 (new location: hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec/family1/c7f5beeb4749452c854751db0e71cf7e_SeqId_49_) 2023-03-05 19:54:08,631 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testMajorCompactionWithMonthlyPolicy__5lrsvat5ipr0bl1djpail23b96s6pidl6s64bjtjvmqtpq8imrivinq8a63811sm/family1/6732698bc8c8474ba0e8e10ca0502483 2023-03-05 19:54:08,633 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-03-05 19:54:08,634 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-05 19:54:08,635 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151120a8bbe18d5d794fc6be68a64ac39fbb4b to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151120a8bbe18d5d794fc6be68a64ac39fbb4b 2023-03-05 19:54:08,637 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151128fbc24177ef654af4bae3a984ed76b29f to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151128fbc24177ef654af4bae3a984ed76b29f 2023-03-05 19:54:08,638 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151130bc06fd1310204b00aa8ed087fb496dba to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151130bc06fd1310204b00aa8ed087fb496dba 2023-03-05 19:54:08,638 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithMonthlyPolicy 3 expired mob files are deleted 2023-03-05 19:54:08,638 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-03-05 19:54:08,638 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-03-05 19:54:08,638 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-03-05 19:54:08,638 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-03-05 19:54:08,639 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-03-05 19:54:08,639 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithMonthlyPolicy', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-05 19:54:08,640 DEBUG [PEWorker-2] locking.LockProcedure(242): UNLOCKED pid=22, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-03-05 19:54:08,641 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=22, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE in 1.9470 sec 2023-03-05 19:54:08,733 INFO [Listener at localhost/40787] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithMonthlyPolicy Thread=263 (was 250) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_260397064_15 at /127.0.0.1:57258 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x18f5725e-shared-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_260397064_15 at /127.0.0.1:57442 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_260397064_15 at /127.0.0.1:57492 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataStreamer for file /user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015123147f48a822b3b45b783ecc39932bfab6a_del java.lang.Object.wait(Native Method) org.apache.hadoop.hdfs.DataStreamer.run(DataStreamer.java:678) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:62957@0x564e1028 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$45/715920721.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_260397064_15 at /127.0.0.1:57504 [Waiting for operation #15] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Parameter Sending Thread #2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_260397064_15 at /127.0.0.1:57376 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x18f5725e-shared-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x18f5725e-shared-pool-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5db36781-e654-c932-96c1-0485fe524a4f/cluster_01318ba7-3e0b-e9a9-4c5c-f222cae89dc3/dfs/data/data1/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x18f5725e-metaLookup-shared--pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: PacketResponder: BP-875493781-172.31.14.131-1678046014457:blk_1073741898_1074, type=LAST_IN_PIPELINE java.lang.Object.wait(Native Method) java.lang.Object.wait(Object.java:502) org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.waitForAckHead(BlockReceiver.java:1327) org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.run(BlockReceiver.java:1399) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:62957@0x564e1028-SendThread(127.0.0.1:62957) sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_260397064_15 at /127.0.0.1:57390 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_260397064_15 at /127.0.0.1:57342 [Waiting for operation #20] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ResponseProcessor for block BP-875493781-172.31.14.131-1678046014457:blk_1073741898_1074 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:118) java.io.FilterInputStream.read(FilterInputStream.java:83) java.io.FilterInputStream.read(FilterInputStream.java:83) org.apache.hadoop.hdfs.protocolPB.PBHelperClient.vintPrefixed(PBHelperClient.java:454) org.apache.hadoop.hdfs.protocol.datatransfer.PipelineAck.readFields(PipelineAck.java:213) org.apache.hadoop.hdfs.DataStreamer$ResponseProcessor.run(DataStreamer.java:1080) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_260397064_15 at /127.0.0.1:57334 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_260397064_15 at /127.0.0.1:57478 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_260397064_15 at /127.0.0.1:57302 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x18f5725e-shared-pool-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:62957@0x564e1028-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: Thread-481 java.lang.Object.wait(Native Method) org.apache.hadoop.hdfs.DataStreamer.run(DataStreamer.java:678) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_260397064_15 at /127.0.0.1:57462 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_260397064_15 at /127.0.0.1:57424 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x18f5725e-shared-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5db36781-e654-c932-96c1-0485fe524a4f/cluster_01318ba7-3e0b-e9a9-4c5c-f222cae89dc3/dfs/data/data2/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x1608c6c-shared-pool-23 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_260397064_15 at /127.0.0.1:57568 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_395068685_15 at /127.0.0.1:57592 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_260397064_15 at /127.0.0.1:57280 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataStreamer for file /user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec/.tmp/family1/042d2dfdbc2e4eb799be3047d63887fd block BP-875493781-172.31.14.131-1678046014457:blk_1073741898_1074 java.lang.Object.wait(Native Method) org.apache.hadoop.hdfs.DataStreamer.run(DataStreamer.java:678) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_395068685_15 at /127.0.0.1:57552 [Receiving block BP-875493781-172.31.14.131-1678046014457:blk_1073741898_1074] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read1(BufferedInputStream.java:286) java.io.BufferedInputStream.read(BufferedInputStream.java:345) java.io.DataInputStream.read(DataInputStream.java:149) org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:209) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:211) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109) org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:528) org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:968) org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:877) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:166) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:103) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:290) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x18f5725e-shared-pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_260397064_15 at /127.0.0.1:57438 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=612 (was 586) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=289 (was 327), ProcessCount=173 (was 171) - ProcessCount LEAK? -, AvailableMemoryMB=5557 (was 4936) - AvailableMemoryMB LEAK? - 2023-03-05 19:54:08,746 INFO [Listener at localhost/40787] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMinorCompactionWithWeeklyPolicy Thread=263, OpenFileDescriptor=612, MaxFileDescriptor=60000, SystemLoadAverage=289, ProcessCount=173, AvailableMemoryMB=5556 2023-03-05 19:54:08,748 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMinorCompactionWithWeeklyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-03-05 19:54:08,750 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure2.ProcedureExecutor(1029): Stored pid=23, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:08,751 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_PRE_OPERATION 2023-03-05 19:54:08,752 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMinorCompactionWithWeeklyPolicy" procId is: 23 2023-03-05 19:54:08,752 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-05 19:54:08,753 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-03-05 19:54:08,754 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/.tmp/data/default/testMinorCompactionWithWeeklyPolicy/e6a63acce9d53b978339ac99badba6fa 2023-03-05 19:54:08,755 DEBUG [HFileArchiver-4] backup.HFileArchiver(153): Directory hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/.tmp/data/default/testMinorCompactionWithWeeklyPolicy/e6a63acce9d53b978339ac99badba6fa empty. 2023-03-05 19:54:08,755 DEBUG [HFileArchiver-4] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/.tmp/data/default/testMinorCompactionWithWeeklyPolicy/e6a63acce9d53b978339ac99badba6fa 2023-03-05 19:54:08,755 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived testMinorCompactionWithWeeklyPolicy regions 2023-03-05 19:54:08,854 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-03-05 19:54:09,055 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-03-05 19:54:09,165 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/.tmp/data/default/testMinorCompactionWithWeeklyPolicy/.tabledesc/.tableinfo.0000000001 2023-03-05 19:54:09,166 INFO [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(7675): creating {ENCODED => e6a63acce9d53b978339ac99badba6fa, NAME => 'testMinorCompactionWithWeeklyPolicy,,1678046048748.e6a63acce9d53b978339ac99badba6fa.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMinorCompactionWithWeeklyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/.tmp 2023-03-05 19:54:09,356 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-03-05 19:54:09,573 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(866): Instantiated testMinorCompactionWithWeeklyPolicy,,1678046048748.e6a63acce9d53b978339ac99badba6fa.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-05 19:54:09,574 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1604): Closing e6a63acce9d53b978339ac99badba6fa, disabling compactions & flushes 2023-03-05 19:54:09,574 INFO [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1626): Closing region testMinorCompactionWithWeeklyPolicy,,1678046048748.e6a63acce9d53b978339ac99badba6fa. 2023-03-05 19:54:09,574 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMinorCompactionWithWeeklyPolicy,,1678046048748.e6a63acce9d53b978339ac99badba6fa. 2023-03-05 19:54:09,574 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1714): Acquired close lock on testMinorCompactionWithWeeklyPolicy,,1678046048748.e6a63acce9d53b978339ac99badba6fa. after waiting 0 ms 2023-03-05 19:54:09,574 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1724): Updates disabled for region testMinorCompactionWithWeeklyPolicy,,1678046048748.e6a63acce9d53b978339ac99badba6fa. 2023-03-05 19:54:09,574 INFO [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1838): Closed testMinorCompactionWithWeeklyPolicy,,1678046048748.e6a63acce9d53b978339ac99badba6fa. 2023-03-05 19:54:09,574 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1558): Region close journal for e6a63acce9d53b978339ac99badba6fa: 2023-03-05 19:54:09,576 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_ADD_TO_META 2023-03-05 19:54:09,577 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMinorCompactionWithWeeklyPolicy,,1678046048748.e6a63acce9d53b978339ac99badba6fa.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1678046049577"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678046049577"}]},"ts":"1678046049577"} 2023-03-05 19:54:09,579 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-03-05 19:54:09,580 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-05 19:54:09,580 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMinorCompactionWithWeeklyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678046049580"}]},"ts":"1678046049580"} 2023-03-05 19:54:09,581 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testMinorCompactionWithWeeklyPolicy, state=ENABLING in hbase:meta 2023-03-05 19:54:09,588 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=24, ppid=23, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMinorCompactionWithWeeklyPolicy, region=e6a63acce9d53b978339ac99badba6fa, ASSIGN}] 2023-03-05 19:54:09,590 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=24, ppid=23, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMinorCompactionWithWeeklyPolicy, region=e6a63acce9d53b978339ac99badba6fa, ASSIGN 2023-03-05 19:54:09,591 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=24, ppid=23, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMinorCompactionWithWeeklyPolicy, region=e6a63acce9d53b978339ac99badba6fa, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,37037,1678046018983; forceNewPlan=false, retain=false 2023-03-05 19:54:09,742 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=24 updating hbase:meta row=e6a63acce9d53b978339ac99badba6fa, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:09,742 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMinorCompactionWithWeeklyPolicy,,1678046048748.e6a63acce9d53b978339ac99badba6fa.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1678046049742"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678046049742"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678046049742"}]},"ts":"1678046049742"} 2023-03-05 19:54:09,744 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=25, ppid=24, state=RUNNABLE; OpenRegionProcedure e6a63acce9d53b978339ac99badba6fa, server=jenkins-hbase4.apache.org,37037,1678046018983}] 2023-03-05 19:54:09,761 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec/.tmp/family1/042d2dfdbc2e4eb799be3047d63887fd as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec/family1/042d2dfdbc2e4eb799be3047d63887fd 2023-03-05 19:54:09,767 INFO [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in aa3a9f808cf1db1e0667a9fff093c3ec/family1 of aa3a9f808cf1db1e0667a9fff093c3ec into 042d2dfdbc2e4eb799be3047d63887fd(size=167.4 K), total size for store is 333.6 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-05 19:54:09,767 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for aa3a9f808cf1db1e0667a9fff093c3ec: 2023-03-05 19:54:09,767 INFO [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithMonthlyPolicy,,1678046035983.aa3a9f808cf1db1e0667a9fff093c3ec., storeName=family1, fileCount=3, fileSize=178.1 K (166.0 K, 6.6 K, 5.5 K), priority=2, time=1678046048507; duration=1sec 2023-03-05 19:54:09,767 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-05 19:54:09,799 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec/family1/bc58a3df1dd0415fbe3b9606b4318973, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec/family1/293e83fcfcac495d88a4f698553f6da5, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec/family1/1fa6b060683a4225934b8332b06b92f6_SeqId_47_] to archive 2023-03-05 19:54:09,800 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-05 19:54:09,802 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec/family1/bc58a3df1dd0415fbe3b9606b4318973 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec/family1/bc58a3df1dd0415fbe3b9606b4318973 2023-03-05 19:54:09,803 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec/family1/293e83fcfcac495d88a4f698553f6da5 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec/family1/293e83fcfcac495d88a4f698553f6da5 2023-03-05 19:54:09,804 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec/family1/1fa6b060683a4225934b8332b06b92f6_SeqId_47_ to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec/family1/1fa6b060683a4225934b8332b06b92f6_SeqId_47_ 2023-03-05 19:54:09,857 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-03-05 19:54:09,899 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMinorCompactionWithWeeklyPolicy,,1678046048748.e6a63acce9d53b978339ac99badba6fa. 2023-03-05 19:54:09,899 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => e6a63acce9d53b978339ac99badba6fa, NAME => 'testMinorCompactionWithWeeklyPolicy,,1678046048748.e6a63acce9d53b978339ac99badba6fa.', STARTKEY => '', ENDKEY => ''} 2023-03-05 19:54:09,899 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompactionWithWeeklyPolicy e6a63acce9d53b978339ac99badba6fa 2023-03-05 19:54:09,899 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMinorCompactionWithWeeklyPolicy,,1678046048748.e6a63acce9d53b978339ac99badba6fa.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-05 19:54:09,899 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for e6a63acce9d53b978339ac99badba6fa 2023-03-05 19:54:09,899 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for e6a63acce9d53b978339ac99badba6fa 2023-03-05 19:54:09,901 INFO [StoreOpener-e6a63acce9d53b978339ac99badba6fa-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region e6a63acce9d53b978339ac99badba6fa 2023-03-05 19:54:09,902 DEBUG [StoreOpener-e6a63acce9d53b978339ac99badba6fa-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithWeeklyPolicy/e6a63acce9d53b978339ac99badba6fa/family1 2023-03-05 19:54:09,902 DEBUG [StoreOpener-e6a63acce9d53b978339ac99badba6fa-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithWeeklyPolicy/e6a63acce9d53b978339ac99badba6fa/family1 2023-03-05 19:54:09,902 INFO [StoreOpener-e6a63acce9d53b978339ac99badba6fa-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e6a63acce9d53b978339ac99badba6fa columnFamilyName family1 2023-03-05 19:54:09,904 INFO [StoreOpener-e6a63acce9d53b978339ac99badba6fa-1] regionserver.HStore(310): Store=e6a63acce9d53b978339ac99badba6fa/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-05 19:54:09,905 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithWeeklyPolicy/e6a63acce9d53b978339ac99badba6fa 2023-03-05 19:54:09,905 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithWeeklyPolicy/e6a63acce9d53b978339ac99badba6fa 2023-03-05 19:54:09,908 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for e6a63acce9d53b978339ac99badba6fa 2023-03-05 19:54:09,910 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithWeeklyPolicy/e6a63acce9d53b978339ac99badba6fa/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-05 19:54:09,910 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened e6a63acce9d53b978339ac99badba6fa; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65025651, jitterRate=-0.031042292714118958}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-05 19:54:09,910 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for e6a63acce9d53b978339ac99badba6fa: 2023-03-05 19:54:09,911 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMinorCompactionWithWeeklyPolicy,,1678046048748.e6a63acce9d53b978339ac99badba6fa., pid=25, masterSystemTime=1678046049896 2023-03-05 19:54:09,913 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMinorCompactionWithWeeklyPolicy,,1678046048748.e6a63acce9d53b978339ac99badba6fa. 2023-03-05 19:54:09,913 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMinorCompactionWithWeeklyPolicy,,1678046048748.e6a63acce9d53b978339ac99badba6fa. 2023-03-05 19:54:09,913 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=24 updating hbase:meta row=e6a63acce9d53b978339ac99badba6fa, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:09,914 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMinorCompactionWithWeeklyPolicy,,1678046048748.e6a63acce9d53b978339ac99badba6fa.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1678046049913"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678046049913"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678046049913"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678046049913"}]},"ts":"1678046049913"} 2023-03-05 19:54:09,917 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=25, resume processing ppid=24 2023-03-05 19:54:09,917 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=25, ppid=24, state=SUCCESS; OpenRegionProcedure e6a63acce9d53b978339ac99badba6fa, server=jenkins-hbase4.apache.org,37037,1678046018983 in 171 msec 2023-03-05 19:54:09,918 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=24, resume processing ppid=23 2023-03-05 19:54:09,919 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=24, ppid=23, state=SUCCESS; TransitRegionStateProcedure table=testMinorCompactionWithWeeklyPolicy, region=e6a63acce9d53b978339ac99badba6fa, ASSIGN in 329 msec 2023-03-05 19:54:09,919 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-05 19:54:09,919 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMinorCompactionWithWeeklyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678046049919"}]},"ts":"1678046049919"} 2023-03-05 19:54:09,921 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testMinorCompactionWithWeeklyPolicy, state=ENABLED in hbase:meta 2023-03-05 19:54:09,923 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_POST_OPERATION 2023-03-05 19:54:09,924 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=23, state=SUCCESS; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy in 1.1750 sec 2023-03-05 19:54:10,859 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-03-05 19:54:10,859 INFO [Listener at localhost/40787] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMinorCompactionWithWeeklyPolicy, procId: 23 completed 2023-03-05 19:54:10,880 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-05 19:54:10,882 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:10,882 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-05 19:54:10,882 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:54:10,882 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-05 19:54:10,882 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-05 19:54:10,882 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:10,882 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:10,883 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:54:10,883 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:10,884 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:54:10,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:54:10,884 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:10,884 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-05 19:54:10,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:10,884 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:10,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-05 19:54:10,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:10,885 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:10,885 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:10,885 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-05 19:54:10,885 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:54:10,885 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-05 19:54:10,885 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-05 19:54:10,885 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-05 19:54:10,885 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1678046048748.e6a63acce9d53b978339ac99badba6fa. 2023-03-05 19:54:10,885 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1678046048748.e6a63acce9d53b978339ac99badba6fa. started... 2023-03-05 19:54:10,886 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing e6a63acce9d53b978339ac99badba6fa 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-03-05 19:54:11,165 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMinorCompactionWithWeeklyPolicy' 2023-03-05 19:54:11,304 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511307348f6fb5e9f4d5f9457bb714523efd1 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201511307348f6fb5e9f4d5f9457bb714523efd1 2023-03-05 19:54:11,708 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithWeeklyPolicy/e6a63acce9d53b978339ac99badba6fa/.tmp/family1/b842f67000fc4759be8bf6f6907e80a4 2023-03-05 19:54:11,714 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithWeeklyPolicy/e6a63acce9d53b978339ac99badba6fa/.tmp/family1/b842f67000fc4759be8bf6f6907e80a4 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithWeeklyPolicy/e6a63acce9d53b978339ac99badba6fa/family1/b842f67000fc4759be8bf6f6907e80a4 2023-03-05 19:54:11,719 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithWeeklyPolicy/e6a63acce9d53b978339ac99badba6fa/family1/b842f67000fc4759be8bf6f6907e80a4, entries=1000, sequenceid=5, filesize=165.1 K 2023-03-05 19:54:11,720 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for e6a63acce9d53b978339ac99badba6fa in 834ms, sequenceid=5, compaction requested=false 2023-03-05 19:54:11,721 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for e6a63acce9d53b978339ac99badba6fa: 2023-03-05 19:54:11,721 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1678046048748.e6a63acce9d53b978339ac99badba6fa. 2023-03-05 19:54:11,721 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-05 19:54:11,721 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-05 19:54:11,721 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:11,721 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-05 19:54:11,721 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-05 19:54:11,724 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:11,724 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:11,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:11,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:11,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:11,724 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:11,724 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-05 19:54:11,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:11,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:11,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:11,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:11,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:11,725 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-05 19:54:11,725 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@22da4b8b[Count = 0] remaining members to acquire global barrier 2023-03-05 19:54:11,725 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-05 19:54:11,725 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:11,727 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:11,727 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:11,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:11,727 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-05 19:54:11,727 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-05 19:54:11,727 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,37037,1678046018983' in zk 2023-03-05 19:54:11,727 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:11,727 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-05 19:54:11,728 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:11,728 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-05 19:54:11,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:11,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:11,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:11,728 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:54:11,728 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-05 19:54:11,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:11,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:11,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:11,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:11,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:11,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:11,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:11,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,37037,1678046018983': 2023-03-05 19:54:11,731 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-05 19:54:11,731 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-05 19:54:11,731 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-05 19:54:11,731 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-05 19:54:11,731 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:11,731 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-05 19:54:11,732 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:11,732 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:11,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:11,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:11,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:11,732 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:11,732 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:54:11,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:11,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:11,733 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:11,733 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:54:11,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:54:11,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:11,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:11,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:11,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:11,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:11,734 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:11,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:11,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:11,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:11,738 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:11,738 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:54:11,738 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:11,738 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:54:11,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:54:11,738 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:54:11,738 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-05 19:54:11,739 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-05 19:54:11,738 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:54:11,738 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:11,739 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:54:11,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:54:11,739 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:11,739 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-05 19:54:11,739 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:11,739 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-05 19:54:11,739 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:11,739 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:11,839 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-05 19:54:11,840 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-05 19:54:11,844 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-05 19:54:11,846 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-05 19:54:11,846 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:11,846 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-05 19:54:11,846 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:54:11,847 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-05 19:54:11,847 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-05 19:54:11,847 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:11,847 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:11,848 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:54:11,848 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:11,848 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:54:11,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:54:11,849 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:11,849 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-05 19:54:11,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:11,849 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:11,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-05 19:54:11,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:11,850 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:11,851 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-03-05 19:54:11,851 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:11,851 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-05 19:54:11,851 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:54:11,851 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-05 19:54:11,851 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-05 19:54:11,851 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-05 19:54:11,851 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1678046048748.e6a63acce9d53b978339ac99badba6fa. 2023-03-05 19:54:11,852 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1678046048748.e6a63acce9d53b978339ac99badba6fa. started... 2023-03-05 19:54:11,852 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing e6a63acce9d53b978339ac99badba6fa 1/1 column families, dataSize=74 B heapSize=408 B 2023-03-05 19:54:12,266 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511288c8d3a338e994b879f6bc92c47dca648 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201511288c8d3a338e994b879f6bc92c47dca648 2023-03-05 19:54:12,672 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithWeeklyPolicy/e6a63acce9d53b978339ac99badba6fa/.tmp/family1/c8c9ed77bf05419d9162aaf790539a45 2023-03-05 19:54:12,677 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithWeeklyPolicy/e6a63acce9d53b978339ac99badba6fa/.tmp/family1/c8c9ed77bf05419d9162aaf790539a45 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithWeeklyPolicy/e6a63acce9d53b978339ac99badba6fa/family1/c8c9ed77bf05419d9162aaf790539a45 2023-03-05 19:54:12,682 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithWeeklyPolicy/e6a63acce9d53b978339ac99badba6fa/family1/c8c9ed77bf05419d9162aaf790539a45, entries=1, sequenceid=9, filesize=5.0 K 2023-03-05 19:54:12,683 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for e6a63acce9d53b978339ac99badba6fa in 831ms, sequenceid=9, compaction requested=false 2023-03-05 19:54:12,683 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for e6a63acce9d53b978339ac99badba6fa: 2023-03-05 19:54:12,683 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1678046048748.e6a63acce9d53b978339ac99badba6fa. 2023-03-05 19:54:12,683 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-05 19:54:12,683 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-05 19:54:12,684 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:12,684 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-05 19:54:12,684 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-05 19:54:12,685 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:12,685 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:12,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:12,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:12,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:12,686 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:12,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:12,686 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-05 19:54:12,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:12,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:12,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:12,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:12,687 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-05 19:54:12,687 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@71a9b48c[Count = 0] remaining members to acquire global barrier 2023-03-05 19:54:12,687 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-05 19:54:12,687 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:12,688 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:12,688 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:12,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:12,688 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-05 19:54:12,689 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-05 19:54:12,689 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,37037,1678046018983' in zk 2023-03-05 19:54:12,689 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:12,689 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-05 19:54:12,691 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:12,691 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-05 19:54:12,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:12,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:12,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:12,691 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:54:12,691 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-05 19:54:12,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:12,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:12,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:12,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:12,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:12,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:12,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:12,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,37037,1678046018983': 2023-03-05 19:54:12,693 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-05 19:54:12,693 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-05 19:54:12,693 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-05 19:54:12,693 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-05 19:54:12,693 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:12,693 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-05 19:54:12,695 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:12,695 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:12,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:12,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:12,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:12,695 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:12,695 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:54:12,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:12,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:12,695 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:54:12,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:54:12,695 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:12,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:12,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:12,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:12,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:12,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:12,696 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:12,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:12,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:12,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:12,698 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:12,699 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:54:12,699 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:12,699 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:54:12,699 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-05 19:54:12,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:54:12,699 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-05 19:54:12,699 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-05 19:54:12,699 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:54:12,699 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:12,699 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:54:12,699 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:12,699 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:54:12,699 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-05 19:54:12,699 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:12,699 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-05 19:54:12,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:54:12,699 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:12,699 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:12,799 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-05 19:54:12,800 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-05 19:54:12,805 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-05 19:54:12,806 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-05 19:54:12,806 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:12,806 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-05 19:54:12,807 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:54:12,807 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-05 19:54:12,807 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-05 19:54:12,807 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:12,807 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:12,808 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:12,808 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:54:12,808 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:54:12,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:54:12,809 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:12,809 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-05 19:54:12,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:12,809 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:12,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-05 19:54:12,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:12,809 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:12,810 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-03-05 19:54:12,810 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:12,810 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-05 19:54:12,810 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:54:12,810 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-05 19:54:12,810 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-05 19:54:12,810 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-05 19:54:12,810 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1678046048748.e6a63acce9d53b978339ac99badba6fa. 2023-03-05 19:54:12,810 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1678046048748.e6a63acce9d53b978339ac99badba6fa. started... 2023-03-05 19:54:12,810 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing e6a63acce9d53b978339ac99badba6fa 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-05 19:54:13,224 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120141e0965462f44c24a465ddad65d60e1c to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2015120141e0965462f44c24a465ddad65d60e1c 2023-03-05 19:54:13,630 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithWeeklyPolicy/e6a63acce9d53b978339ac99badba6fa/.tmp/family1/429c3245e7854d91bcedd751de1c4d92 2023-03-05 19:54:13,636 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithWeeklyPolicy/e6a63acce9d53b978339ac99badba6fa/.tmp/family1/429c3245e7854d91bcedd751de1c4d92 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithWeeklyPolicy/e6a63acce9d53b978339ac99badba6fa/family1/429c3245e7854d91bcedd751de1c4d92 2023-03-05 19:54:13,640 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithWeeklyPolicy/e6a63acce9d53b978339ac99badba6fa/family1/429c3245e7854d91bcedd751de1c4d92, entries=1, sequenceid=13, filesize=4.9 K 2023-03-05 19:54:13,641 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for e6a63acce9d53b978339ac99badba6fa in 831ms, sequenceid=13, compaction requested=true 2023-03-05 19:54:13,641 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for e6a63acce9d53b978339ac99badba6fa: 2023-03-05 19:54:13,641 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1678046048748.e6a63acce9d53b978339ac99badba6fa. 2023-03-05 19:54:13,641 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-05 19:54:13,641 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-05 19:54:13,641 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:13,641 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-05 19:54:13,641 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-05 19:54:13,644 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:13,644 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:13,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:13,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:13,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:13,644 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:13,644 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-05 19:54:13,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:13,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:13,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:13,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:13,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:13,645 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-05 19:54:13,645 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@16fd1798[Count = 0] remaining members to acquire global barrier 2023-03-05 19:54:13,645 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-05 19:54:13,646 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:13,647 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:13,647 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:13,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:13,647 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-05 19:54:13,647 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-05 19:54:13,647 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,37037,1678046018983' in zk 2023-03-05 19:54:13,647 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:13,647 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-05 19:54:13,648 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-05 19:54:13,648 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:54:13,648 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:13,648 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-05 19:54:13,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:13,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:13,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:13,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:13,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:13,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:13,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:13,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:13,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:13,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:13,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,37037,1678046018983': 2023-03-05 19:54:13,650 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-05 19:54:13,650 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-05 19:54:13,650 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-05 19:54:13,650 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-05 19:54:13,650 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:13,650 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-05 19:54:13,652 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:13,652 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:13,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:13,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:13,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:13,652 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:13,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:13,652 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:54:13,652 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:54:13,652 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:13,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:54:13,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:13,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:13,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:13,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:13,653 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:13,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:13,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:13,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:13,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:13,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:13,659 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:13,659 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:13,659 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:13,659 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:54:13,659 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-05 19:54:13,659 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:54:13,659 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:54:13,659 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:54:13,659 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:13,659 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-05 19:54:13,659 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:13,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:54:13,659 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-05 19:54:13,659 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:13,659 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:13,660 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:54:13,660 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-05 19:54:13,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:54:13,660 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-05 19:54:13,760 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-05 19:54:13,761 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-05 19:54:13,765 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-05 19:54:13,766 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-05 19:54:13,766 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:13,766 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-05 19:54:13,766 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:54:13,767 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-05 19:54:13,767 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-05 19:54:13,767 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:13,767 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:13,768 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:54:13,768 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:13,768 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:54:13,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:54:13,769 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:13,769 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-05 19:54:13,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:13,769 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:13,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-05 19:54:13,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:13,769 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:13,770 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-03-05 19:54:13,770 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:13,770 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-05 19:54:13,770 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:54:13,770 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-05 19:54:13,770 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-05 19:54:13,770 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-05 19:54:13,770 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1678046048748.e6a63acce9d53b978339ac99badba6fa. 2023-03-05 19:54:13,770 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1678046048748.e6a63acce9d53b978339ac99badba6fa. started... 2023-03-05 19:54:13,770 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing e6a63acce9d53b978339ac99badba6fa 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-05 19:54:14,183 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120582573b53997442ea88b88e69f5fed0f6 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2015120582573b53997442ea88b88e69f5fed0f6 2023-03-05 19:54:14,588 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithWeeklyPolicy/e6a63acce9d53b978339ac99badba6fa/.tmp/family1/50f10123b4af41fcbdd6bf9dd9eb0659 2023-03-05 19:54:14,594 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithWeeklyPolicy/e6a63acce9d53b978339ac99badba6fa/.tmp/family1/50f10123b4af41fcbdd6bf9dd9eb0659 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithWeeklyPolicy/e6a63acce9d53b978339ac99badba6fa/family1/50f10123b4af41fcbdd6bf9dd9eb0659 2023-03-05 19:54:14,598 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithWeeklyPolicy/e6a63acce9d53b978339ac99badba6fa/family1/50f10123b4af41fcbdd6bf9dd9eb0659, entries=1, sequenceid=17, filesize=4.9 K 2023-03-05 19:54:14,599 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for e6a63acce9d53b978339ac99badba6fa in 829ms, sequenceid=17, compaction requested=true 2023-03-05 19:54:14,599 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for e6a63acce9d53b978339ac99badba6fa: 2023-03-05 19:54:14,599 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1678046048748.e6a63acce9d53b978339ac99badba6fa. 2023-03-05 19:54:14,599 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-05 19:54:14,599 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-05 19:54:14,599 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:14,600 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-05 19:54:14,600 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-05 19:54:14,601 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:14,601 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:14,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:14,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:14,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:14,602 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:14,602 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-05 19:54:14,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:14,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:14,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:14,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:14,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:14,603 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-05 19:54:14,603 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@646b7a8f[Count = 0] remaining members to acquire global barrier 2023-03-05 19:54:14,603 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-05 19:54:14,603 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:14,604 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:14,604 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:14,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:14,604 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-05 19:54:14,604 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-05 19:54:14,605 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:14,605 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-05 19:54:14,605 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,37037,1678046018983' in zk 2023-03-05 19:54:14,606 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:14,606 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-05 19:54:14,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:14,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:14,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:14,606 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:54:14,606 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-05 19:54:14,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:14,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:14,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:14,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:14,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:14,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:14,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:14,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,37037,1678046018983': 2023-03-05 19:54:14,608 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-05 19:54:14,608 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-05 19:54:14,608 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-05 19:54:14,608 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-05 19:54:14,608 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:14,608 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-05 19:54:14,611 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:14,611 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:14,611 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:14,611 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:54:14,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:14,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:14,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:14,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:14,611 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:14,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:14,611 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:54:14,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:54:14,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:14,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:14,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:14,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:14,612 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:14,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:14,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:14,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:14,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:14,614 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:14,615 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:54:14,615 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:14,615 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:54:14,615 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-05 19:54:14,615 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-05 19:54:14,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:54:14,615 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:54:14,615 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:14,615 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:54:14,615 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:14,615 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-05 19:54:14,615 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:14,615 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-05 19:54:14,615 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:14,615 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:14,615 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:54:14,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:54:14,715 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-05 19:54:14,716 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-05 19:54:14,720 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-05 19:54:14,721 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-05 19:54:14,722 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:14,722 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-05 19:54:14,722 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:54:14,722 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-05 19:54:14,722 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-05 19:54:14,723 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:14,723 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:14,724 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:54:14,724 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:14,724 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:54:14,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:54:14,724 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:14,724 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-05 19:54:14,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:14,725 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:14,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-05 19:54:14,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:14,725 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:14,725 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-03-05 19:54:14,725 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:14,725 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-05 19:54:14,725 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:54:14,725 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-05 19:54:14,725 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-05 19:54:14,725 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-05 19:54:14,725 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1678046048748.e6a63acce9d53b978339ac99badba6fa. 2023-03-05 19:54:14,726 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1678046048748.e6a63acce9d53b978339ac99badba6fa. started... 2023-03-05 19:54:14,726 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing e6a63acce9d53b978339ac99badba6fa 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-05 19:54:15,139 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151228a2ecfad536b945a6b7dc44bb7e7ed1be to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151228a2ecfad536b945a6b7dc44bb7e7ed1be 2023-03-05 19:54:15,425 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-05 19:54:15,545 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithWeeklyPolicy/e6a63acce9d53b978339ac99badba6fa/.tmp/family1/5d37c21050c540dbb961c0d75bc99267 2023-03-05 19:54:15,551 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithWeeklyPolicy/e6a63acce9d53b978339ac99badba6fa/.tmp/family1/5d37c21050c540dbb961c0d75bc99267 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithWeeklyPolicy/e6a63acce9d53b978339ac99badba6fa/family1/5d37c21050c540dbb961c0d75bc99267 2023-03-05 19:54:15,555 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithWeeklyPolicy/e6a63acce9d53b978339ac99badba6fa/family1/5d37c21050c540dbb961c0d75bc99267, entries=1, sequenceid=21, filesize=4.9 K 2023-03-05 19:54:15,556 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for e6a63acce9d53b978339ac99badba6fa in 830ms, sequenceid=21, compaction requested=true 2023-03-05 19:54:15,556 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for e6a63acce9d53b978339ac99badba6fa: 2023-03-05 19:54:15,556 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1678046048748.e6a63acce9d53b978339ac99badba6fa. 2023-03-05 19:54:15,556 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-05 19:54:15,556 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-05 19:54:15,556 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:15,556 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-05 19:54:15,556 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-05 19:54:15,558 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:15,558 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:15,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:15,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:15,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:15,558 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:15,558 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-05 19:54:15,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:15,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:15,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:15,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:15,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:15,559 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-05 19:54:15,560 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@149de39a[Count = 0] remaining members to acquire global barrier 2023-03-05 19:54:15,560 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-05 19:54:15,560 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:15,562 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:15,562 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:15,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:15,562 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-05 19:54:15,562 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-05 19:54:15,562 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:15,562 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-05 19:54:15,562 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,37037,1678046018983' in zk 2023-03-05 19:54:15,564 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:15,564 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-05 19:54:15,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:15,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:15,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:15,564 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:54:15,564 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-05 19:54:15,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:15,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:15,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:15,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:15,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:15,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:15,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:15,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,37037,1678046018983': 2023-03-05 19:54:15,566 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-05 19:54:15,566 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-05 19:54:15,566 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-05 19:54:15,566 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-05 19:54:15,566 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:15,566 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-05 19:54:15,568 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:15,568 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:15,568 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:15,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:15,568 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:54:15,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:15,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:15,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:15,568 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:54:15,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:54:15,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:15,568 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:15,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:15,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:15,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:15,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:15,569 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:15,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:15,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:15,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:15,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:15,571 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:15,571 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:15,571 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:54:15,571 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:15,571 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:54:15,571 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-05 19:54:15,571 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:15,571 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:54:15,571 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-05 19:54:15,572 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-05 19:54:15,571 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:54:15,572 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:15,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:54:15,572 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:15,572 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:15,572 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-05 19:54:15,572 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-05 19:54:15,572 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:54:15,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:54:15,672 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-05 19:54:15,673 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-05 19:54:15,677 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-05 19:54:15,679 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-05 19:54:15,679 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:15,679 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-05 19:54:15,679 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:54:15,679 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-05 19:54:15,679 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-05 19:54:15,680 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:15,680 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:15,682 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:54:15,682 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:15,682 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:54:15,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:54:15,683 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:15,683 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-05 19:54:15,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:15,683 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:15,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-05 19:54:15,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:15,683 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:15,683 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-03-05 19:54:15,684 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:15,684 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-05 19:54:15,684 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:54:15,684 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-05 19:54:15,684 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-05 19:54:15,684 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-05 19:54:15,684 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1678046048748.e6a63acce9d53b978339ac99badba6fa. 2023-03-05 19:54:15,684 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1678046048748.e6a63acce9d53b978339ac99badba6fa. started... 2023-03-05 19:54:15,684 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing e6a63acce9d53b978339ac99badba6fa 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-05 19:54:16,098 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231de14a9ae5ac6468a960a0ea25dce679a to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151231de14a9ae5ac6468a960a0ea25dce679a 2023-03-05 19:54:16,504 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithWeeklyPolicy/e6a63acce9d53b978339ac99badba6fa/.tmp/family1/a2401cdfc5554c0bafabd25179e87600 2023-03-05 19:54:16,511 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithWeeklyPolicy/e6a63acce9d53b978339ac99badba6fa/.tmp/family1/a2401cdfc5554c0bafabd25179e87600 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithWeeklyPolicy/e6a63acce9d53b978339ac99badba6fa/family1/a2401cdfc5554c0bafabd25179e87600 2023-03-05 19:54:16,518 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithWeeklyPolicy/e6a63acce9d53b978339ac99badba6fa/family1/a2401cdfc5554c0bafabd25179e87600, entries=1, sequenceid=25, filesize=4.9 K 2023-03-05 19:54:16,519 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for e6a63acce9d53b978339ac99badba6fa in 835ms, sequenceid=25, compaction requested=true 2023-03-05 19:54:16,519 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for e6a63acce9d53b978339ac99badba6fa: 2023-03-05 19:54:16,519 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1678046048748.e6a63acce9d53b978339ac99badba6fa. 2023-03-05 19:54:16,519 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-05 19:54:16,519 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-05 19:54:16,519 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:16,519 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-05 19:54:16,519 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-05 19:54:16,521 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:16,521 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:16,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:16,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:16,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:16,522 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:16,522 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-05 19:54:16,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:16,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:16,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:16,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:16,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:16,524 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-05 19:54:16,524 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@28ac3851[Count = 0] remaining members to acquire global barrier 2023-03-05 19:54:16,524 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-05 19:54:16,524 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:16,525 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:16,525 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:16,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:16,525 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-05 19:54:16,525 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-05 19:54:16,525 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,37037,1678046018983' in zk 2023-03-05 19:54:16,526 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:16,526 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-05 19:54:16,527 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:16,527 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-05 19:54:16,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:16,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:16,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:16,527 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:54:16,528 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-05 19:54:16,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:16,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:16,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:16,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:16,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:16,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:16,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:16,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,37037,1678046018983': 2023-03-05 19:54:16,530 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-05 19:54:16,530 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-05 19:54:16,530 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-05 19:54:16,530 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-05 19:54:16,530 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:16,530 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-05 19:54:16,535 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:16,535 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:16,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:16,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:16,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:16,535 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:16,535 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:54:16,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:16,536 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:16,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:16,536 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:54:16,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:54:16,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:16,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:16,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:16,537 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:16,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:16,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:16,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:16,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:16,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:16,539 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:16,539 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:54:16,539 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:16,539 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-05 19:54:16,540 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-05 19:54:16,539 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:16,539 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:54:16,539 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:54:16,539 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:54:16,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:54:16,540 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-05 19:54:16,540 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:16,540 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-05 19:54:16,540 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-05 19:54:16,540 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:54:16,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:54:16,540 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:16,540 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:16,540 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:16,640 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-05 19:54:16,641 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-05 19:54:16,646 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-05 19:54:16,648 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-05 19:54:16,648 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:16,648 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-05 19:54:16,648 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:54:16,648 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-05 19:54:16,648 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-05 19:54:16,649 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:16,649 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:16,650 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:16,650 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:54:16,650 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:54:16,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:54:16,651 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:16,651 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-05 19:54:16,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:16,651 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:16,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-05 19:54:16,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:16,652 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:16,653 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-03-05 19:54:16,653 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:16,653 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-05 19:54:16,653 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:54:16,653 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-05 19:54:16,653 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-05 19:54:16,653 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-05 19:54:16,653 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1678046048748.e6a63acce9d53b978339ac99badba6fa. 2023-03-05 19:54:16,653 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1678046048748.e6a63acce9d53b978339ac99badba6fa. started... 2023-03-05 19:54:16,654 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing e6a63acce9d53b978339ac99badba6fa 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-05 19:54:17,069 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2016010121b578bf751041cab2e1a57da1b943ee to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2016010121b578bf751041cab2e1a57da1b943ee 2023-03-05 19:54:17,476 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithWeeklyPolicy/e6a63acce9d53b978339ac99badba6fa/.tmp/family1/469c4e84e431450cb1e247282bd2673a 2023-03-05 19:54:17,483 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithWeeklyPolicy/e6a63acce9d53b978339ac99badba6fa/.tmp/family1/469c4e84e431450cb1e247282bd2673a as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithWeeklyPolicy/e6a63acce9d53b978339ac99badba6fa/family1/469c4e84e431450cb1e247282bd2673a 2023-03-05 19:54:17,489 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithWeeklyPolicy/e6a63acce9d53b978339ac99badba6fa/family1/469c4e84e431450cb1e247282bd2673a, entries=1, sequenceid=29, filesize=4.9 K 2023-03-05 19:54:17,490 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for e6a63acce9d53b978339ac99badba6fa in 836ms, sequenceid=29, compaction requested=true 2023-03-05 19:54:17,490 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for e6a63acce9d53b978339ac99badba6fa: 2023-03-05 19:54:17,490 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1678046048748.e6a63acce9d53b978339ac99badba6fa. 2023-03-05 19:54:17,490 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-05 19:54:17,490 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-05 19:54:17,490 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:17,490 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-05 19:54:17,490 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-05 19:54:17,492 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:17,492 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:17,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:17,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:17,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:17,492 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:17,492 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-05 19:54:17,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:17,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:17,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:17,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:17,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:17,494 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-05 19:54:17,494 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5693ab0b[Count = 0] remaining members to acquire global barrier 2023-03-05 19:54:17,494 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-05 19:54:17,494 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:17,496 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:17,496 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:17,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:17,497 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-05 19:54:17,497 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-05 19:54:17,497 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,37037,1678046018983' in zk 2023-03-05 19:54:17,497 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:17,497 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-05 19:54:17,498 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-05 19:54:17,498 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:17,498 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:54:17,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:17,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:17,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:17,498 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-05 19:54:17,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:17,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:17,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:17,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:17,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:17,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:17,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:17,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,37037,1678046018983': 2023-03-05 19:54:17,501 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-05 19:54:17,501 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-05 19:54:17,501 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-05 19:54:17,501 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-05 19:54:17,501 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:17,501 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-05 19:54:17,502 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:17,502 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:17,502 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:17,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:17,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:17,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:17,502 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:54:17,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:17,503 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:17,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:17,503 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:54:17,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:54:17,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:17,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:17,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:17,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:17,504 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:17,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:17,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:17,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:17,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:17,506 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:54:17,506 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:17,506 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:54:17,506 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:17,506 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:54:17,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:54:17,506 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:54:17,507 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:17,506 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-05 19:54:17,506 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-05 19:54:17,507 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-05 19:54:17,507 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:54:17,507 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:17,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:54:17,507 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:17,507 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:17,507 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:17,507 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-05 19:54:17,507 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-05 19:54:17,607 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-05 19:54:17,609 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-05 19:54:17,613 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-05 19:54:17,615 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-05 19:54:17,615 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:17,615 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-05 19:54:17,615 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:54:17,615 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-05 19:54:17,615 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-05 19:54:17,616 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:17,616 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:17,618 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:54:17,618 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:17,619 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:54:17,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:54:17,619 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:17,619 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-05 19:54:17,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:17,619 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:17,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-05 19:54:17,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:17,620 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:17,620 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-03-05 19:54:17,620 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:17,620 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-05 19:54:17,620 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:54:17,620 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-05 19:54:17,620 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-05 19:54:17,620 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-05 19:54:17,620 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1678046048748.e6a63acce9d53b978339ac99badba6fa. 2023-03-05 19:54:17,620 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1678046048748.e6a63acce9d53b978339ac99badba6fa. started... 2023-03-05 19:54:17,621 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing e6a63acce9d53b978339ac99badba6fa 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-05 19:54:18,036 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601030c4f083b27b646258848c62bcd83e005 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201601030c4f083b27b646258848c62bcd83e005 2023-03-05 19:54:18,442 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithWeeklyPolicy/e6a63acce9d53b978339ac99badba6fa/.tmp/family1/35c52cba4adc4125809b2d2cab9bf4c0 2023-03-05 19:54:18,448 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithWeeklyPolicy/e6a63acce9d53b978339ac99badba6fa/.tmp/family1/35c52cba4adc4125809b2d2cab9bf4c0 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithWeeklyPolicy/e6a63acce9d53b978339ac99badba6fa/family1/35c52cba4adc4125809b2d2cab9bf4c0 2023-03-05 19:54:18,453 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithWeeklyPolicy/e6a63acce9d53b978339ac99badba6fa/family1/35c52cba4adc4125809b2d2cab9bf4c0, entries=1, sequenceid=33, filesize=4.9 K 2023-03-05 19:54:18,454 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for e6a63acce9d53b978339ac99badba6fa in 833ms, sequenceid=33, compaction requested=true 2023-03-05 19:54:18,454 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for e6a63acce9d53b978339ac99badba6fa: 2023-03-05 19:54:18,454 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1678046048748.e6a63acce9d53b978339ac99badba6fa. 2023-03-05 19:54:18,454 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-05 19:54:18,454 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-05 19:54:18,454 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:18,454 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-05 19:54:18,454 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-05 19:54:18,456 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:18,456 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:18,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:18,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:18,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:18,456 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:18,456 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-05 19:54:18,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:18,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:18,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:18,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:18,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:18,458 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-05 19:54:18,458 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@69131c54[Count = 0] remaining members to acquire global barrier 2023-03-05 19:54:18,458 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-05 19:54:18,458 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:18,460 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:18,460 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:18,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:18,460 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-05 19:54:18,460 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-05 19:54:18,460 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,37037,1678046018983' in zk 2023-03-05 19:54:18,460 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:18,461 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-05 19:54:18,462 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:18,462 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-05 19:54:18,462 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:54:18,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:18,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:18,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:18,462 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-05 19:54:18,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:18,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:18,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:18,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:18,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:18,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:18,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:18,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,37037,1678046018983': 2023-03-05 19:54:18,464 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-05 19:54:18,464 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-05 19:54:18,465 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-05 19:54:18,465 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-05 19:54:18,465 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:18,465 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-05 19:54:18,466 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:18,466 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:18,466 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:18,466 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:54:18,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:18,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:18,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:18,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:18,467 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:54:18,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:54:18,467 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:18,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:18,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:18,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:18,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:18,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:18,468 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:18,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:18,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:18,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:18,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:18,470 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:18,470 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:18,470 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:54:18,471 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-05 19:54:18,470 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:54:18,470 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:18,470 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-05 19:54:18,471 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-05 19:54:18,471 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:18,471 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:54:18,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:54:18,471 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:54:18,471 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:18,471 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:18,471 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-05 19:54:18,471 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-05 19:54:18,471 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:18,471 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:54:18,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:54:18,571 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-05 19:54:18,573 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-05 19:54:18,578 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-05 19:54:18,579 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-05 19:54:18,580 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:18,580 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-05 19:54:18,580 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:54:18,580 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-05 19:54:18,580 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-05 19:54:18,580 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:18,580 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:18,583 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:54:18,583 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:18,583 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:54:18,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:54:18,583 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:18,583 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-05 19:54:18,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:18,584 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:18,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-05 19:54:18,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:18,584 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:18,585 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-03-05 19:54:18,585 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:18,585 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-05 19:54:18,585 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:54:18,585 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-05 19:54:18,585 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-05 19:54:18,585 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-05 19:54:18,585 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1678046048748.e6a63acce9d53b978339ac99badba6fa. 2023-03-05 19:54:18,585 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1678046048748.e6a63acce9d53b978339ac99badba6fa. started... 2023-03-05 19:54:18,585 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing e6a63acce9d53b978339ac99badba6fa 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-05 19:54:19,009 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201509073d984254dd8e452a918192122ef3b9ab to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201509073d984254dd8e452a918192122ef3b9ab 2023-03-05 19:54:19,014 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithWeeklyPolicy/e6a63acce9d53b978339ac99badba6fa/.tmp/family1/a923f106e1334108bffa6a5703c65f24 2023-03-05 19:54:19,020 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithWeeklyPolicy/e6a63acce9d53b978339ac99badba6fa/.tmp/family1/a923f106e1334108bffa6a5703c65f24 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithWeeklyPolicy/e6a63acce9d53b978339ac99badba6fa/family1/a923f106e1334108bffa6a5703c65f24 2023-03-05 19:54:19,026 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithWeeklyPolicy/e6a63acce9d53b978339ac99badba6fa/family1/a923f106e1334108bffa6a5703c65f24, entries=1, sequenceid=37, filesize=4.9 K 2023-03-05 19:54:19,031 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for e6a63acce9d53b978339ac99badba6fa in 446ms, sequenceid=37, compaction requested=true 2023-03-05 19:54:19,031 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for e6a63acce9d53b978339ac99badba6fa: 2023-03-05 19:54:19,031 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1678046048748.e6a63acce9d53b978339ac99badba6fa. 2023-03-05 19:54:19,031 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-05 19:54:19,031 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-05 19:54:19,031 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:19,031 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-05 19:54:19,031 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-05 19:54:19,033 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:19,033 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:19,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:19,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:19,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:19,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:19,034 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:19,034 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-05 19:54:19,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:19,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:19,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:19,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:19,035 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-05 19:54:19,035 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1269a3df[Count = 0] remaining members to acquire global barrier 2023-03-05 19:54:19,035 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-05 19:54:19,036 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:19,040 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:19,040 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:19,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:19,040 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-05 19:54:19,040 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-05 19:54:19,040 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,37037,1678046018983' in zk 2023-03-05 19:54:19,041 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:19,041 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-05 19:54:19,042 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:19,042 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-05 19:54:19,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:19,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:19,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:19,043 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:54:19,043 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-05 19:54:19,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:19,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:19,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:19,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:19,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:19,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:19,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:19,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,37037,1678046018983': 2023-03-05 19:54:19,045 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-05 19:54:19,045 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-05 19:54:19,045 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-05 19:54:19,045 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-05 19:54:19,045 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:19,045 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-05 19:54:19,047 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:19,047 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:19,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:19,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:19,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:19,047 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:19,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:19,047 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:54:19,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:19,048 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:54:19,048 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:19,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:54:19,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:19,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:19,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:19,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:19,049 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:19,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:19,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:19,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:19,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:19,052 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:19,052 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:54:19,052 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:19,052 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:54:19,052 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:19,052 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:54:19,052 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-05 19:54:19,053 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-05 19:54:19,052 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:54:19,052 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:19,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:54:19,053 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-05 19:54:19,053 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:54:19,053 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:19,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:54:19,053 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-05 19:54:19,053 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:19,054 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:19,154 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-05 19:54:19,155 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-05 19:54:19,161 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-05 19:54:19,169 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-05 19:54:19,169 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:19,169 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-05 19:54:19,169 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:54:19,169 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-05 19:54:19,170 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-05 19:54:19,170 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:19,170 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:19,171 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:19,172 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:54:19,172 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:54:19,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:54:19,172 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:19,172 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-05 19:54:19,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:19,172 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:19,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-05 19:54:19,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:19,173 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:19,173 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-03-05 19:54:19,173 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:19,173 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-05 19:54:19,173 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:54:19,173 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-05 19:54:19,174 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-05 19:54:19,174 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-05 19:54:19,174 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1678046048748.e6a63acce9d53b978339ac99badba6fa. 2023-03-05 19:54:19,174 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1678046048748.e6a63acce9d53b978339ac99badba6fa. started... 2023-03-05 19:54:19,174 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing e6a63acce9d53b978339ac99badba6fa 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-05 19:54:19,188 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015112078100e3ec4354c4b9351996b6de509d4 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2015112078100e3ec4354c4b9351996b6de509d4 2023-03-05 19:54:19,594 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithWeeklyPolicy/e6a63acce9d53b978339ac99badba6fa/.tmp/family1/b0279adb06e6484a98d7cb76848adfa0 2023-03-05 19:54:19,605 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithWeeklyPolicy/e6a63acce9d53b978339ac99badba6fa/.tmp/family1/b0279adb06e6484a98d7cb76848adfa0 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithWeeklyPolicy/e6a63acce9d53b978339ac99badba6fa/family1/b0279adb06e6484a98d7cb76848adfa0 2023-03-05 19:54:19,611 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithWeeklyPolicy/e6a63acce9d53b978339ac99badba6fa/family1/b0279adb06e6484a98d7cb76848adfa0, entries=1, sequenceid=41, filesize=4.9 K 2023-03-05 19:54:19,612 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for e6a63acce9d53b978339ac99badba6fa in 438ms, sequenceid=41, compaction requested=true 2023-03-05 19:54:19,612 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for e6a63acce9d53b978339ac99badba6fa: 2023-03-05 19:54:19,612 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1678046048748.e6a63acce9d53b978339ac99badba6fa. 2023-03-05 19:54:19,612 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-05 19:54:19,612 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-05 19:54:19,612 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:19,612 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-05 19:54:19,612 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-05 19:54:19,614 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:19,614 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:19,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:19,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:19,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:19,614 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:19,614 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-05 19:54:19,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:19,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:19,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:19,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:19,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:19,616 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-05 19:54:19,616 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5b4e6f73[Count = 0] remaining members to acquire global barrier 2023-03-05 19:54:19,616 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-05 19:54:19,616 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:19,617 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:19,617 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:19,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:19,617 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:19,618 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-05 19:54:19,617 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-05 19:54:19,618 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-05 19:54:19,618 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,37037,1678046018983' in zk 2023-03-05 19:54:19,620 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:19,620 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-05 19:54:19,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:19,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:19,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:19,620 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:54:19,620 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-05 19:54:19,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:19,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:19,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:19,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:19,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:19,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:19,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:19,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,37037,1678046018983': 2023-03-05 19:54:19,623 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-05 19:54:19,623 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-05 19:54:19,623 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-05 19:54:19,623 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-05 19:54:19,623 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:19,623 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-05 19:54:19,626 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:19,626 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:19,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:19,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:19,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:19,626 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:19,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:19,626 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:54:19,626 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:19,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:19,627 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:54:19,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:54:19,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:19,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:19,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:19,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:19,628 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:19,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:19,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:19,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:19,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:19,631 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:19,631 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:19,631 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:19,632 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:19,632 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:19,632 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:54:19,632 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-05 19:54:19,632 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-05 19:54:19,632 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:19,632 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:19,632 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:54:19,632 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:54:19,632 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:54:19,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:54:19,632 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-05 19:54:19,633 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-05 19:54:19,633 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:54:19,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:54:19,635 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-05 19:54:19,733 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-05 19:54:19,734 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-05 19:54:19,735 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=38221] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMinorCompactionWithWeeklyPolicy 2023-03-05 19:54:19,737 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] procedure2.ProcedureExecutor(1029): Stored pid=26, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-03-05 19:54:19,738 DEBUG [PEWorker-1] locking.LockProcedure(309): LOCKED pid=26, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-03-05 19:54:19,739 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] mob.MobUtils(833): start MOB compaction of files for table='testMinorCompactionWithWeeklyPolicy', column='family1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-05 19:54:19,739 INFO [PEWorker-1] procedure2.TimeoutExecutorThread(81): ADDED pid=26, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678046659739 2023-03-05 19:54:19,742 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-03-05 19:54:19,743 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] compactions.PartitionedMobCompactor(306): The compaction type is PART_FILES, the request has 0 del files, 9 selected files, and 0 irrelevant files table 'testMinorCompactionWithWeeklyPolicy' and column 'family1' 2023-03-05 19:54:19,743 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-03-05 19:54:19,744 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] zookeeper.ReadOnlyZKClient(139): Connect 0x2c113d25 to 127.0.0.1:62957 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-05 19:54:19,752 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1f45cd7b, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-05 19:54:19,752 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-05 19:54:20,602 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103f2214972920942318d9e03c95f565f2a to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20160103f2214972920942318d9e03c95f565f2a 2023-03-05 19:54:20,603 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-03-05 19:54:20,616 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-05 19:54:20,617 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:54110, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-05 19:54:20,623 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-05 19:54:20,625 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:50180, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-05 19:54:20,634 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/e99600d7a3ef4898ab23565da4801f6f first=Optional[r3] last=Optional[r6] 2023-03-05 19:54:20,634 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMinorCompactionWithWeeklyPolicy,,1678046048748.e6a63acce9d53b978339ac99badba6fa., hostname=jenkins-hbase4.apache.org,37037,1678046018983, seqNum=2 for row with hfile group [{family1,hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/e99600d7a3ef4898ab23565da4801f6f}] 2023-03-05 19:54:20,639 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(763): Validating hfile at hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/e99600d7a3ef4898ab23565da4801f6f for inclusion in e6a63acce9d53b978339ac99badba6fa/family1 2023-03-05 19:54:20,642 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-03-05 19:54:20,642 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(777): Region bounds: first= last= 2023-03-05 19:54:20,642 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HRegion(2446): Flush status journal for e6a63acce9d53b978339ac99badba6fa: 2023-03-05 19:54:20,643 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/e99600d7a3ef4898ab23565da4801f6f to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testMinorCompactionWithWeeklyPolicy__oms1p5rj1qks2n40ol4kjcephppoos1e0ieiq4dsv12f94oc16rjarke9oeveq50/family1/e99600d7a3ef4898ab23565da4801f6f 2023-03-05 19:54:20,646 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testMinorCompactionWithWeeklyPolicy__oms1p5rj1qks2n40ol4kjcephppoos1e0ieiq4dsv12f94oc16rjarke9oeveq50/family1/e99600d7a3ef4898ab23565da4801f6f as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithWeeklyPolicy/e6a63acce9d53b978339ac99badba6fa/family1/5bb4805658e3475bbe082a31fa57702f_SeqId_44_ 2023-03-05 19:54:20,646 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(849): Loaded HFile hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testMinorCompactionWithWeeklyPolicy__oms1p5rj1qks2n40ol4kjcephppoos1e0ieiq4dsv12f94oc16rjarke9oeveq50/family1/e99600d7a3ef4898ab23565da4801f6f into e6a63acce9d53b978339ac99badba6fa/family1 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithWeeklyPolicy/e6a63acce9d53b978339ac99badba6fa/family1/5bb4805658e3475bbe082a31fa57702f_SeqId_44_ - updating store file list. 2023-03-05 19:54:20,651 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(882): Loaded HFile hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithWeeklyPolicy/e6a63acce9d53b978339ac99badba6fa/family1/5bb4805658e3475bbe082a31fa57702f_SeqId_44_ into e6a63acce9d53b978339ac99badba6fa/family1 2023-03-05 19:54:20,651 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(855): Successfully loaded hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testMinorCompactionWithWeeklyPolicy__oms1p5rj1qks2n40ol4kjcephppoos1e0ieiq4dsv12f94oc16rjarke9oeveq50/family1/e99600d7a3ef4898ab23565da4801f6f into e6a63acce9d53b978339ac99badba6fa/family1 (new location: hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithWeeklyPolicy/e6a63acce9d53b978339ac99badba6fa/family1/5bb4805658e3475bbe082a31fa57702f_SeqId_44_) 2023-03-05 19:54:20,652 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testMinorCompactionWithWeeklyPolicy__oms1p5rj1qks2n40ol4kjcephppoos1e0ieiq4dsv12f94oc16rjarke9oeveq50/family1/e99600d7a3ef4898ab23565da4801f6f 2023-03-05 19:54:20,652 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-03-05 19:54:20,656 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51214 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-03-05 19:54:20,656 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(1912): e6a63acce9d53b978339ac99badba6fa/family1 is initiating minor compaction 2023-03-05 19:54:20,656 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@630c1745; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-05 19:54:20,656 INFO [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of e6a63acce9d53b978339ac99badba6fa/family1 in testMinorCompactionWithWeeklyPolicy,,1678046048748.e6a63acce9d53b978339ac99badba6fa. 2023-03-05 19:54:20,657 INFO [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithWeeklyPolicy/e6a63acce9d53b978339ac99badba6fa/family1/c8c9ed77bf05419d9162aaf790539a45, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithWeeklyPolicy/e6a63acce9d53b978339ac99badba6fa/family1/429c3245e7854d91bcedd751de1c4d92, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithWeeklyPolicy/e6a63acce9d53b978339ac99badba6fa/family1/50f10123b4af41fcbdd6bf9dd9eb0659, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithWeeklyPolicy/e6a63acce9d53b978339ac99badba6fa/family1/5d37c21050c540dbb961c0d75bc99267, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithWeeklyPolicy/e6a63acce9d53b978339ac99badba6fa/family1/a2401cdfc5554c0bafabd25179e87600, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithWeeklyPolicy/e6a63acce9d53b978339ac99badba6fa/family1/469c4e84e431450cb1e247282bd2673a, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithWeeklyPolicy/e6a63acce9d53b978339ac99badba6fa/family1/35c52cba4adc4125809b2d2cab9bf4c0, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithWeeklyPolicy/e6a63acce9d53b978339ac99badba6fa/family1/a923f106e1334108bffa6a5703c65f24, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithWeeklyPolicy/e6a63acce9d53b978339ac99badba6fa/family1/b0279adb06e6484a98d7cb76848adfa0, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithWeeklyPolicy/e6a63acce9d53b978339ac99badba6fa/family1/5bb4805658e3475bbe082a31fa57702f_SeqId_44_] into tmpdir=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithWeeklyPolicy/e6a63acce9d53b978339ac99badba6fa/.tmp, totalSize=50.0 K 2023-03-05 19:54:20,657 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.Compactor(207): Compacting c8c9ed77bf05419d9162aaf790539a45, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-03-05 19:54:20,657 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.Compactor(207): Compacting 429c3245e7854d91bcedd751de1c4d92, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=13 2023-03-05 19:54:20,657 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.Compactor(207): Compacting 50f10123b4af41fcbdd6bf9dd9eb0659, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=17 2023-03-05 19:54:20,657 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.Compactor(207): Compacting 5d37c21050c540dbb961c0d75bc99267, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=21 2023-03-05 19:54:20,657 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.Compactor(207): Compacting a2401cdfc5554c0bafabd25179e87600, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=25 2023-03-05 19:54:20,657 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.Compactor(207): Compacting 469c4e84e431450cb1e247282bd2673a, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=29 2023-03-05 19:54:20,657 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.Compactor(207): Compacting 35c52cba4adc4125809b2d2cab9bf4c0, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=33 2023-03-05 19:54:20,657 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.Compactor(207): Compacting a923f106e1334108bffa6a5703c65f24, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=37 2023-03-05 19:54:20,658 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.Compactor(207): Compacting b0279adb06e6484a98d7cb76848adfa0, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=41 2023-03-05 19:54:20,658 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.Compactor(207): Compacting 5bb4805658e3475bbe082a31fa57702f_SeqId_44_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=45 2023-03-05 19:54:20,658 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-03-05 19:54:20,659 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-05 19:54:20,661 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151228a2ecfad536b945a6b7dc44bb7e7ed1be to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151228a2ecfad536b945a6b7dc44bb7e7ed1be 2023-03-05 19:54:20,662 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151231de14a9ae5ac6468a960a0ea25dce679a to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151231de14a9ae5ac6468a960a0ea25dce679a 2023-03-05 19:54:20,664 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2016010121b578bf751041cab2e1a57da1b943ee to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2016010121b578bf751041cab2e1a57da1b943ee 2023-03-05 19:54:20,665 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201601030c4f083b27b646258848c62bcd83e005 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201601030c4f083b27b646258848c62bcd83e005 2023-03-05 19:54:20,666 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithWeeklyPolicy 4 expired mob files are deleted 2023-03-05 19:54:20,666 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-03-05 19:54:20,666 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-05 19:54:20,666 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-05 19:54:20,666 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-05 19:54:20,666 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-05 19:54:20,666 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-05 19:54:20,666 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-05 19:54:20,666 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-05 19:54:20,666 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-05 19:54:20,703 INFO [RS:0;jenkins-hbase4:37037-shortCompactions-0] throttle.PressureAwareThroughputController(145): e6a63acce9d53b978339ac99badba6fa#family1#compaction#34 average throughput is 0.45 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-05 19:54:21,118 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512058dfd6e54985c4f7cb69430a39e3c577a to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201512058dfd6e54985c4f7cb69430a39e3c577a 2023-03-05 19:54:21,119 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-03-05 19:54:21,147 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/084f088b77124449adf319417137bd04 first=Optional[r1] last=Optional[r2] 2023-03-05 19:54:21,147 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMinorCompactionWithWeeklyPolicy,,1678046048748.e6a63acce9d53b978339ac99badba6fa., hostname=jenkins-hbase4.apache.org,37037,1678046018983, seqNum=2 for row with hfile group [{family1,hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/084f088b77124449adf319417137bd04}] 2023-03-05 19:54:21,150 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(763): Validating hfile at hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/084f088b77124449adf319417137bd04 for inclusion in e6a63acce9d53b978339ac99badba6fa/family1 2023-03-05 19:54:21,153 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(775): HFile bounds: first=r1 last=r2 2023-03-05 19:54:21,153 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(777): Region bounds: first= last= 2023-03-05 19:54:21,154 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HRegion(2446): Flush status journal for e6a63acce9d53b978339ac99badba6fa: 2023-03-05 19:54:21,155 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/084f088b77124449adf319417137bd04 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testMinorCompactionWithWeeklyPolicy__fi6a0rpv94vq6ilmsrjbikajgnl4v1l6dv38otqr5ljcd1m442fgnd4iqr7b22im/family1/084f088b77124449adf319417137bd04 2023-03-05 19:54:21,157 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testMinorCompactionWithWeeklyPolicy__fi6a0rpv94vq6ilmsrjbikajgnl4v1l6dv38otqr5ljcd1m442fgnd4iqr7b22im/family1/084f088b77124449adf319417137bd04 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithWeeklyPolicy/e6a63acce9d53b978339ac99badba6fa/family1/be17f1200c92415487aac43ef7b0625a_SeqId_46_ 2023-03-05 19:54:21,158 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(849): Loaded HFile hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testMinorCompactionWithWeeklyPolicy__fi6a0rpv94vq6ilmsrjbikajgnl4v1l6dv38otqr5ljcd1m442fgnd4iqr7b22im/family1/084f088b77124449adf319417137bd04 into e6a63acce9d53b978339ac99badba6fa/family1 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithWeeklyPolicy/e6a63acce9d53b978339ac99badba6fa/family1/be17f1200c92415487aac43ef7b0625a_SeqId_46_ - updating store file list. 2023-03-05 19:54:21,162 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(882): Loaded HFile hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithWeeklyPolicy/e6a63acce9d53b978339ac99badba6fa/family1/be17f1200c92415487aac43ef7b0625a_SeqId_46_ into e6a63acce9d53b978339ac99badba6fa/family1 2023-03-05 19:54:21,162 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(855): Successfully loaded hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testMinorCompactionWithWeeklyPolicy__fi6a0rpv94vq6ilmsrjbikajgnl4v1l6dv38otqr5ljcd1m442fgnd4iqr7b22im/family1/084f088b77124449adf319417137bd04 into e6a63acce9d53b978339ac99badba6fa/family1 (new location: hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithWeeklyPolicy/e6a63acce9d53b978339ac99badba6fa/family1/be17f1200c92415487aac43ef7b0625a_SeqId_46_) 2023-03-05 19:54:21,163 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testMinorCompactionWithWeeklyPolicy__fi6a0rpv94vq6ilmsrjbikajgnl4v1l6dv38otqr5ljcd1m442fgnd4iqr7b22im/family1/084f088b77124449adf319417137bd04 2023-03-05 19:54:21,165 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-03-05 19:54:21,165 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-05 19:54:21,167 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2015120141e0965462f44c24a465ddad65d60e1c to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2015120141e0965462f44c24a465ddad65d60e1c 2023-03-05 19:54:21,168 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2015120582573b53997442ea88b88e69f5fed0f6 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2015120582573b53997442ea88b88e69f5fed0f6 2023-03-05 19:54:21,168 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithWeeklyPolicy 2 expired mob files are deleted 2023-03-05 19:54:21,168 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-03-05 19:54:21,169 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-05 19:54:21,169 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-03-05 19:54:21,169 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] mob.MobUtils(840): end MOB compaction of files for table='testMinorCompactionWithWeeklyPolicy', column='family1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-05 19:54:21,170 DEBUG [PEWorker-5] locking.LockProcedure(242): UNLOCKED pid=26, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-03-05 19:54:21,171 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=26, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE in 1.4340 sec 2023-03-05 19:54:21,249 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-05 19:54:21,249 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x32fab577 to 127.0.0.1:62957 2023-03-05 19:54:21,249 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-05 19:54:21,251 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-05 19:54:21,253 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x2c113d25 to 127.0.0.1:62957 2023-03-05 19:54:21,253 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-05 19:54:21,253 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-05 19:54:21,253 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x564e1028 to 127.0.0.1:62957 2023-03-05 19:54:21,253 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-05 19:54:21,254 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-05 19:54:21,254 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x3ed3bf19 to 127.0.0.1:62957 2023-03-05 19:54:21,254 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-05 19:54:21,304 INFO [Listener at localhost/40787] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMinorCompactionWithWeeklyPolicy Thread=255 (was 263), OpenFileDescriptor=604 (was 612), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=277 (was 289), ProcessCount=173 (was 173), AvailableMemoryMB=5241 (was 5556) 2023-03-05 19:54:21,320 INFO [Listener at localhost/40787] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testScannerOnBulkLoadRefHFiles Thread=255, OpenFileDescriptor=604, MaxFileDescriptor=60000, SystemLoadAverage=277, ProcessCount=173, AvailableMemoryMB=5227 2023-03-05 19:54:21,330 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testScannerOnBulkLoadRefHFiles', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-03-05 19:54:21,332 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure2.ProcedureExecutor(1029): Stored pid=27, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles 2023-03-05 19:54:21,335 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_PRE_OPERATION 2023-03-05 19:54:21,335 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testScannerOnBulkLoadRefHFiles" procId is: 27 2023-03-05 19:54:21,336 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-05 19:54:21,336 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-03-05 19:54:21,338 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/.tmp/data/default/testScannerOnBulkLoadRefHFiles/237c9aa4b41fcfa45032528ec18afe9b 2023-03-05 19:54:21,338 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/.tmp/data/default/testScannerOnBulkLoadRefHFiles/319680d28f8da52640b97eff4eaa9239 2023-03-05 19:54:21,341 DEBUG [HFileArchiver-5] backup.HFileArchiver(153): Directory hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/.tmp/data/default/testScannerOnBulkLoadRefHFiles/237c9aa4b41fcfa45032528ec18afe9b empty. 2023-03-05 19:54:21,341 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/.tmp/data/default/testScannerOnBulkLoadRefHFiles/2d86bbeed3970cf2266d199ff776cf8d 2023-03-05 19:54:21,341 DEBUG [HFileArchiver-6] backup.HFileArchiver(153): Directory hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/.tmp/data/default/testScannerOnBulkLoadRefHFiles/319680d28f8da52640b97eff4eaa9239 empty. 2023-03-05 19:54:21,341 DEBUG [HFileArchiver-5] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/.tmp/data/default/testScannerOnBulkLoadRefHFiles/237c9aa4b41fcfa45032528ec18afe9b 2023-03-05 19:54:21,341 DEBUG [HFileArchiver-7] backup.HFileArchiver(153): Directory hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/.tmp/data/default/testScannerOnBulkLoadRefHFiles/2d86bbeed3970cf2266d199ff776cf8d empty. 2023-03-05 19:54:21,341 DEBUG [HFileArchiver-6] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/.tmp/data/default/testScannerOnBulkLoadRefHFiles/319680d28f8da52640b97eff4eaa9239 2023-03-05 19:54:21,342 DEBUG [HFileArchiver-7] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/.tmp/data/default/testScannerOnBulkLoadRefHFiles/2d86bbeed3970cf2266d199ff776cf8d 2023-03-05 19:54:21,342 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived testScannerOnBulkLoadRefHFiles regions 2023-03-05 19:54:21,437 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-03-05 19:54:21,519 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithWeeklyPolicy/e6a63acce9d53b978339ac99badba6fa/.tmp/family1/9364d29b37f24104b903f17c57082666 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithWeeklyPolicy/e6a63acce9d53b978339ac99badba6fa/family1/9364d29b37f24104b903f17c57082666 2023-03-05 19:54:21,525 INFO [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in e6a63acce9d53b978339ac99badba6fa/family1 of e6a63acce9d53b978339ac99badba6fa into 9364d29b37f24104b903f17c57082666(size=6.6 K), total size for store is 176.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-05 19:54:21,525 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for e6a63acce9d53b978339ac99badba6fa: 2023-03-05 19:54:21,525 INFO [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMinorCompactionWithWeeklyPolicy,,1678046048748.e6a63acce9d53b978339ac99badba6fa., storeName=family1, fileCount=10, fileSize=50.0 K (5.0 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 5.5 K), priority=2, time=1678046060656; duration=0sec 2023-03-05 19:54:21,526 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-05 19:54:21,605 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithWeeklyPolicy/e6a63acce9d53b978339ac99badba6fa/family1/c8c9ed77bf05419d9162aaf790539a45, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithWeeklyPolicy/e6a63acce9d53b978339ac99badba6fa/family1/429c3245e7854d91bcedd751de1c4d92, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithWeeklyPolicy/e6a63acce9d53b978339ac99badba6fa/family1/50f10123b4af41fcbdd6bf9dd9eb0659, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithWeeklyPolicy/e6a63acce9d53b978339ac99badba6fa/family1/5d37c21050c540dbb961c0d75bc99267, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithWeeklyPolicy/e6a63acce9d53b978339ac99badba6fa/family1/a2401cdfc5554c0bafabd25179e87600, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithWeeklyPolicy/e6a63acce9d53b978339ac99badba6fa/family1/469c4e84e431450cb1e247282bd2673a, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithWeeklyPolicy/e6a63acce9d53b978339ac99badba6fa/family1/35c52cba4adc4125809b2d2cab9bf4c0, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithWeeklyPolicy/e6a63acce9d53b978339ac99badba6fa/family1/a923f106e1334108bffa6a5703c65f24, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithWeeklyPolicy/e6a63acce9d53b978339ac99badba6fa/family1/b0279adb06e6484a98d7cb76848adfa0, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithWeeklyPolicy/e6a63acce9d53b978339ac99badba6fa/family1/5bb4805658e3475bbe082a31fa57702f_SeqId_44_] to archive 2023-03-05 19:54:21,606 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-05 19:54:21,608 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithWeeklyPolicy/e6a63acce9d53b978339ac99badba6fa/family1/c8c9ed77bf05419d9162aaf790539a45 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMinorCompactionWithWeeklyPolicy/e6a63acce9d53b978339ac99badba6fa/family1/c8c9ed77bf05419d9162aaf790539a45 2023-03-05 19:54:21,609 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithWeeklyPolicy/e6a63acce9d53b978339ac99badba6fa/family1/429c3245e7854d91bcedd751de1c4d92 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMinorCompactionWithWeeklyPolicy/e6a63acce9d53b978339ac99badba6fa/family1/429c3245e7854d91bcedd751de1c4d92 2023-03-05 19:54:21,610 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithWeeklyPolicy/e6a63acce9d53b978339ac99badba6fa/family1/50f10123b4af41fcbdd6bf9dd9eb0659 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMinorCompactionWithWeeklyPolicy/e6a63acce9d53b978339ac99badba6fa/family1/50f10123b4af41fcbdd6bf9dd9eb0659 2023-03-05 19:54:21,611 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithWeeklyPolicy/e6a63acce9d53b978339ac99badba6fa/family1/5d37c21050c540dbb961c0d75bc99267 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMinorCompactionWithWeeklyPolicy/e6a63acce9d53b978339ac99badba6fa/family1/5d37c21050c540dbb961c0d75bc99267 2023-03-05 19:54:21,612 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithWeeklyPolicy/e6a63acce9d53b978339ac99badba6fa/family1/a2401cdfc5554c0bafabd25179e87600 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMinorCompactionWithWeeklyPolicy/e6a63acce9d53b978339ac99badba6fa/family1/a2401cdfc5554c0bafabd25179e87600 2023-03-05 19:54:21,613 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithWeeklyPolicy/e6a63acce9d53b978339ac99badba6fa/family1/469c4e84e431450cb1e247282bd2673a to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMinorCompactionWithWeeklyPolicy/e6a63acce9d53b978339ac99badba6fa/family1/469c4e84e431450cb1e247282bd2673a 2023-03-05 19:54:21,614 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithWeeklyPolicy/e6a63acce9d53b978339ac99badba6fa/family1/35c52cba4adc4125809b2d2cab9bf4c0 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMinorCompactionWithWeeklyPolicy/e6a63acce9d53b978339ac99badba6fa/family1/35c52cba4adc4125809b2d2cab9bf4c0 2023-03-05 19:54:21,615 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithWeeklyPolicy/e6a63acce9d53b978339ac99badba6fa/family1/a923f106e1334108bffa6a5703c65f24 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMinorCompactionWithWeeklyPolicy/e6a63acce9d53b978339ac99badba6fa/family1/a923f106e1334108bffa6a5703c65f24 2023-03-05 19:54:21,616 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithWeeklyPolicy/e6a63acce9d53b978339ac99badba6fa/family1/b0279adb06e6484a98d7cb76848adfa0 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMinorCompactionWithWeeklyPolicy/e6a63acce9d53b978339ac99badba6fa/family1/b0279adb06e6484a98d7cb76848adfa0 2023-03-05 19:54:21,617 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithWeeklyPolicy/e6a63acce9d53b978339ac99badba6fa/family1/5bb4805658e3475bbe082a31fa57702f_SeqId_44_ to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMinorCompactionWithWeeklyPolicy/e6a63acce9d53b978339ac99badba6fa/family1/5bb4805658e3475bbe082a31fa57702f_SeqId_44_ 2023-03-05 19:54:21,638 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-03-05 19:54:21,751 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/.tmp/data/default/testScannerOnBulkLoadRefHFiles/.tabledesc/.tableinfo.0000000001 2023-03-05 19:54:21,753 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(7675): creating {ENCODED => 2d86bbeed3970cf2266d199ff776cf8d, NAME => 'testScannerOnBulkLoadRefHFiles,2,1678046061330.2d86bbeed3970cf2266d199ff776cf8d.', STARTKEY => '2', ENDKEY => ''}, tableDescriptor='testScannerOnBulkLoadRefHFiles', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/.tmp 2023-03-05 19:54:21,753 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(7675): creating {ENCODED => 237c9aa4b41fcfa45032528ec18afe9b, NAME => 'testScannerOnBulkLoadRefHFiles,,1678046061330.237c9aa4b41fcfa45032528ec18afe9b.', STARTKEY => '', ENDKEY => '1'}, tableDescriptor='testScannerOnBulkLoadRefHFiles', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/.tmp 2023-03-05 19:54:21,753 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(7675): creating {ENCODED => 319680d28f8da52640b97eff4eaa9239, NAME => 'testScannerOnBulkLoadRefHFiles,1,1678046061330.319680d28f8da52640b97eff4eaa9239.', STARTKEY => '1', ENDKEY => '2'}, tableDescriptor='testScannerOnBulkLoadRefHFiles', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/.tmp 2023-03-05 19:54:21,940 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-03-05 19:54:22,171 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(866): Instantiated testScannerOnBulkLoadRefHFiles,2,1678046061330.2d86bbeed3970cf2266d199ff776cf8d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-05 19:54:22,171 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(866): Instantiated testScannerOnBulkLoadRefHFiles,1,1678046061330.319680d28f8da52640b97eff4eaa9239.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-05 19:54:22,171 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1604): Closing 2d86bbeed3970cf2266d199ff776cf8d, disabling compactions & flushes 2023-03-05 19:54:22,172 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1604): Closing 319680d28f8da52640b97eff4eaa9239, disabling compactions & flushes 2023-03-05 19:54:22,172 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1626): Closing region testScannerOnBulkLoadRefHFiles,2,1678046061330.2d86bbeed3970cf2266d199ff776cf8d. 2023-03-05 19:54:22,172 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1647): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,2,1678046061330.2d86bbeed3970cf2266d199ff776cf8d. 2023-03-05 19:54:22,172 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1714): Acquired close lock on testScannerOnBulkLoadRefHFiles,2,1678046061330.2d86bbeed3970cf2266d199ff776cf8d. after waiting 0 ms 2023-03-05 19:54:22,172 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1724): Updates disabled for region testScannerOnBulkLoadRefHFiles,2,1678046061330.2d86bbeed3970cf2266d199ff776cf8d. 2023-03-05 19:54:22,172 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1838): Closed testScannerOnBulkLoadRefHFiles,2,1678046061330.2d86bbeed3970cf2266d199ff776cf8d. 2023-03-05 19:54:22,172 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1558): Region close journal for 2d86bbeed3970cf2266d199ff776cf8d: 2023-03-05 19:54:22,172 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(866): Instantiated testScannerOnBulkLoadRefHFiles,,1678046061330.237c9aa4b41fcfa45032528ec18afe9b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-05 19:54:22,172 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1626): Closing region testScannerOnBulkLoadRefHFiles,1,1678046061330.319680d28f8da52640b97eff4eaa9239. 2023-03-05 19:54:22,172 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1604): Closing 237c9aa4b41fcfa45032528ec18afe9b, disabling compactions & flushes 2023-03-05 19:54:22,173 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1647): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,1,1678046061330.319680d28f8da52640b97eff4eaa9239. 2023-03-05 19:54:22,173 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1626): Closing region testScannerOnBulkLoadRefHFiles,,1678046061330.237c9aa4b41fcfa45032528ec18afe9b. 2023-03-05 19:54:22,173 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,,1678046061330.237c9aa4b41fcfa45032528ec18afe9b. 2023-03-05 19:54:22,173 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1714): Acquired close lock on testScannerOnBulkLoadRefHFiles,1,1678046061330.319680d28f8da52640b97eff4eaa9239. after waiting 0 ms 2023-03-05 19:54:22,173 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1714): Acquired close lock on testScannerOnBulkLoadRefHFiles,,1678046061330.237c9aa4b41fcfa45032528ec18afe9b. after waiting 0 ms 2023-03-05 19:54:22,173 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1724): Updates disabled for region testScannerOnBulkLoadRefHFiles,,1678046061330.237c9aa4b41fcfa45032528ec18afe9b. 2023-03-05 19:54:22,173 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1724): Updates disabled for region testScannerOnBulkLoadRefHFiles,1,1678046061330.319680d28f8da52640b97eff4eaa9239. 2023-03-05 19:54:22,173 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1838): Closed testScannerOnBulkLoadRefHFiles,,1678046061330.237c9aa4b41fcfa45032528ec18afe9b. 2023-03-05 19:54:22,173 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1838): Closed testScannerOnBulkLoadRefHFiles,1,1678046061330.319680d28f8da52640b97eff4eaa9239. 2023-03-05 19:54:22,173 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1558): Region close journal for 237c9aa4b41fcfa45032528ec18afe9b: 2023-03-05 19:54:22,173 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1558): Region close journal for 319680d28f8da52640b97eff4eaa9239: 2023-03-05 19:54:22,175 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_ADD_TO_META 2023-03-05 19:54:22,177 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testScannerOnBulkLoadRefHFiles,2,1678046061330.2d86bbeed3970cf2266d199ff776cf8d.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1678046062176"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678046062176"}]},"ts":"1678046062176"} 2023-03-05 19:54:22,177 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testScannerOnBulkLoadRefHFiles,,1678046061330.237c9aa4b41fcfa45032528ec18afe9b.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1678046062176"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678046062176"}]},"ts":"1678046062176"} 2023-03-05 19:54:22,177 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testScannerOnBulkLoadRefHFiles,1,1678046061330.319680d28f8da52640b97eff4eaa9239.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1678046062176"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678046062176"}]},"ts":"1678046062176"} 2023-03-05 19:54:22,179 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-03-05 19:54:22,180 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-05 19:54:22,180 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testScannerOnBulkLoadRefHFiles","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678046062180"}]},"ts":"1678046062180"} 2023-03-05 19:54:22,182 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testScannerOnBulkLoadRefHFiles, state=ENABLING in hbase:meta 2023-03-05 19:54:22,186 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=237c9aa4b41fcfa45032528ec18afe9b, ASSIGN}, {pid=29, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=319680d28f8da52640b97eff4eaa9239, ASSIGN}, {pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=2d86bbeed3970cf2266d199ff776cf8d, ASSIGN}] 2023-03-05 19:54:22,188 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=2d86bbeed3970cf2266d199ff776cf8d, ASSIGN 2023-03-05 19:54:22,188 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=319680d28f8da52640b97eff4eaa9239, ASSIGN 2023-03-05 19:54:22,188 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=237c9aa4b41fcfa45032528ec18afe9b, ASSIGN 2023-03-05 19:54:22,189 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=2d86bbeed3970cf2266d199ff776cf8d, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,37037,1678046018983; forceNewPlan=false, retain=false 2023-03-05 19:54:22,189 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=29, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=319680d28f8da52640b97eff4eaa9239, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,37037,1678046018983; forceNewPlan=false, retain=false 2023-03-05 19:54:22,189 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=237c9aa4b41fcfa45032528ec18afe9b, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,37037,1678046018983; forceNewPlan=false, retain=false 2023-03-05 19:54:22,341 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=30 updating hbase:meta row=2d86bbeed3970cf2266d199ff776cf8d, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:22,341 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=319680d28f8da52640b97eff4eaa9239, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:22,341 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=28 updating hbase:meta row=237c9aa4b41fcfa45032528ec18afe9b, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:22,341 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testScannerOnBulkLoadRefHFiles,2,1678046061330.2d86bbeed3970cf2266d199ff776cf8d.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1678046062341"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678046062341"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678046062341"}]},"ts":"1678046062341"} 2023-03-05 19:54:22,341 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testScannerOnBulkLoadRefHFiles,,1678046061330.237c9aa4b41fcfa45032528ec18afe9b.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1678046062341"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678046062341"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678046062341"}]},"ts":"1678046062341"} 2023-03-05 19:54:22,341 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testScannerOnBulkLoadRefHFiles,1,1678046061330.319680d28f8da52640b97eff4eaa9239.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1678046062341"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678046062341"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678046062341"}]},"ts":"1678046062341"} 2023-03-05 19:54:22,343 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=31, ppid=30, state=RUNNABLE; OpenRegionProcedure 2d86bbeed3970cf2266d199ff776cf8d, server=jenkins-hbase4.apache.org,37037,1678046018983}] 2023-03-05 19:54:22,344 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=32, ppid=29, state=RUNNABLE; OpenRegionProcedure 319680d28f8da52640b97eff4eaa9239, server=jenkins-hbase4.apache.org,37037,1678046018983}] 2023-03-05 19:54:22,345 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=33, ppid=28, state=RUNNABLE; OpenRegionProcedure 237c9aa4b41fcfa45032528ec18afe9b, server=jenkins-hbase4.apache.org,37037,1678046018983}] 2023-03-05 19:54:22,441 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-03-05 19:54:22,499 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testScannerOnBulkLoadRefHFiles,,1678046061330.237c9aa4b41fcfa45032528ec18afe9b. 2023-03-05 19:54:22,499 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 237c9aa4b41fcfa45032528ec18afe9b, NAME => 'testScannerOnBulkLoadRefHFiles,,1678046061330.237c9aa4b41fcfa45032528ec18afe9b.', STARTKEY => '', ENDKEY => '1'} 2023-03-05 19:54:22,500 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testScannerOnBulkLoadRefHFiles 237c9aa4b41fcfa45032528ec18afe9b 2023-03-05 19:54:22,500 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testScannerOnBulkLoadRefHFiles,,1678046061330.237c9aa4b41fcfa45032528ec18afe9b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-05 19:54:22,500 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 237c9aa4b41fcfa45032528ec18afe9b 2023-03-05 19:54:22,500 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 237c9aa4b41fcfa45032528ec18afe9b 2023-03-05 19:54:22,501 INFO [StoreOpener-237c9aa4b41fcfa45032528ec18afe9b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 237c9aa4b41fcfa45032528ec18afe9b 2023-03-05 19:54:22,503 DEBUG [StoreOpener-237c9aa4b41fcfa45032528ec18afe9b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testScannerOnBulkLoadRefHFiles/237c9aa4b41fcfa45032528ec18afe9b/family1 2023-03-05 19:54:22,503 DEBUG [StoreOpener-237c9aa4b41fcfa45032528ec18afe9b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testScannerOnBulkLoadRefHFiles/237c9aa4b41fcfa45032528ec18afe9b/family1 2023-03-05 19:54:22,504 INFO [StoreOpener-237c9aa4b41fcfa45032528ec18afe9b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 237c9aa4b41fcfa45032528ec18afe9b columnFamilyName family1 2023-03-05 19:54:22,505 INFO [StoreOpener-237c9aa4b41fcfa45032528ec18afe9b-1] regionserver.HStore(310): Store=237c9aa4b41fcfa45032528ec18afe9b/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-05 19:54:22,506 INFO [StoreOpener-237c9aa4b41fcfa45032528ec18afe9b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 237c9aa4b41fcfa45032528ec18afe9b 2023-03-05 19:54:22,507 DEBUG [StoreOpener-237c9aa4b41fcfa45032528ec18afe9b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testScannerOnBulkLoadRefHFiles/237c9aa4b41fcfa45032528ec18afe9b/family2 2023-03-05 19:54:22,507 DEBUG [StoreOpener-237c9aa4b41fcfa45032528ec18afe9b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testScannerOnBulkLoadRefHFiles/237c9aa4b41fcfa45032528ec18afe9b/family2 2023-03-05 19:54:22,507 INFO [StoreOpener-237c9aa4b41fcfa45032528ec18afe9b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 237c9aa4b41fcfa45032528ec18afe9b columnFamilyName family2 2023-03-05 19:54:22,509 INFO [StoreOpener-237c9aa4b41fcfa45032528ec18afe9b-1] regionserver.HStore(310): Store=237c9aa4b41fcfa45032528ec18afe9b/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-05 19:54:22,510 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testScannerOnBulkLoadRefHFiles/237c9aa4b41fcfa45032528ec18afe9b 2023-03-05 19:54:22,510 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testScannerOnBulkLoadRefHFiles/237c9aa4b41fcfa45032528ec18afe9b 2023-03-05 19:54:22,512 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testScannerOnBulkLoadRefHFiles descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-05 19:54:22,513 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 237c9aa4b41fcfa45032528ec18afe9b 2023-03-05 19:54:22,516 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testScannerOnBulkLoadRefHFiles/237c9aa4b41fcfa45032528ec18afe9b/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-05 19:54:22,516 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 237c9aa4b41fcfa45032528ec18afe9b; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66209438, jitterRate=-0.013402491807937622}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-05 19:54:22,516 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 237c9aa4b41fcfa45032528ec18afe9b: 2023-03-05 19:54:22,517 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testScannerOnBulkLoadRefHFiles,,1678046061330.237c9aa4b41fcfa45032528ec18afe9b., pid=33, masterSystemTime=1678046062495 2023-03-05 19:54:22,519 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testScannerOnBulkLoadRefHFiles,,1678046061330.237c9aa4b41fcfa45032528ec18afe9b. 2023-03-05 19:54:22,519 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testScannerOnBulkLoadRefHFiles,,1678046061330.237c9aa4b41fcfa45032528ec18afe9b. 2023-03-05 19:54:22,519 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testScannerOnBulkLoadRefHFiles,1,1678046061330.319680d28f8da52640b97eff4eaa9239. 2023-03-05 19:54:22,519 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 319680d28f8da52640b97eff4eaa9239, NAME => 'testScannerOnBulkLoadRefHFiles,1,1678046061330.319680d28f8da52640b97eff4eaa9239.', STARTKEY => '1', ENDKEY => '2'} 2023-03-05 19:54:22,520 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=28 updating hbase:meta row=237c9aa4b41fcfa45032528ec18afe9b, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:22,520 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testScannerOnBulkLoadRefHFiles 319680d28f8da52640b97eff4eaa9239 2023-03-05 19:54:22,520 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testScannerOnBulkLoadRefHFiles,,1678046061330.237c9aa4b41fcfa45032528ec18afe9b.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1678046062520"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678046062520"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678046062520"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678046062520"}]},"ts":"1678046062520"} 2023-03-05 19:54:22,520 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testScannerOnBulkLoadRefHFiles,1,1678046061330.319680d28f8da52640b97eff4eaa9239.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-05 19:54:22,520 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 319680d28f8da52640b97eff4eaa9239 2023-03-05 19:54:22,520 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 319680d28f8da52640b97eff4eaa9239 2023-03-05 19:54:22,523 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=33, resume processing ppid=28 2023-03-05 19:54:22,523 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=33, ppid=28, state=SUCCESS; OpenRegionProcedure 237c9aa4b41fcfa45032528ec18afe9b, server=jenkins-hbase4.apache.org,37037,1678046018983 in 176 msec 2023-03-05 19:54:22,524 INFO [StoreOpener-319680d28f8da52640b97eff4eaa9239-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 319680d28f8da52640b97eff4eaa9239 2023-03-05 19:54:22,524 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=28, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=237c9aa4b41fcfa45032528ec18afe9b, ASSIGN in 337 msec 2023-03-05 19:54:22,525 DEBUG [StoreOpener-319680d28f8da52640b97eff4eaa9239-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testScannerOnBulkLoadRefHFiles/319680d28f8da52640b97eff4eaa9239/family1 2023-03-05 19:54:22,526 DEBUG [StoreOpener-319680d28f8da52640b97eff4eaa9239-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testScannerOnBulkLoadRefHFiles/319680d28f8da52640b97eff4eaa9239/family1 2023-03-05 19:54:22,526 INFO [StoreOpener-319680d28f8da52640b97eff4eaa9239-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 319680d28f8da52640b97eff4eaa9239 columnFamilyName family1 2023-03-05 19:54:22,527 INFO [StoreOpener-319680d28f8da52640b97eff4eaa9239-1] regionserver.HStore(310): Store=319680d28f8da52640b97eff4eaa9239/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-05 19:54:22,527 INFO [StoreOpener-319680d28f8da52640b97eff4eaa9239-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 319680d28f8da52640b97eff4eaa9239 2023-03-05 19:54:22,529 DEBUG [StoreOpener-319680d28f8da52640b97eff4eaa9239-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testScannerOnBulkLoadRefHFiles/319680d28f8da52640b97eff4eaa9239/family2 2023-03-05 19:54:22,529 DEBUG [StoreOpener-319680d28f8da52640b97eff4eaa9239-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testScannerOnBulkLoadRefHFiles/319680d28f8da52640b97eff4eaa9239/family2 2023-03-05 19:54:22,530 INFO [StoreOpener-319680d28f8da52640b97eff4eaa9239-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 319680d28f8da52640b97eff4eaa9239 columnFamilyName family2 2023-03-05 19:54:22,531 INFO [StoreOpener-319680d28f8da52640b97eff4eaa9239-1] regionserver.HStore(310): Store=319680d28f8da52640b97eff4eaa9239/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-05 19:54:22,532 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testScannerOnBulkLoadRefHFiles/319680d28f8da52640b97eff4eaa9239 2023-03-05 19:54:22,532 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testScannerOnBulkLoadRefHFiles/319680d28f8da52640b97eff4eaa9239 2023-03-05 19:54:22,534 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testScannerOnBulkLoadRefHFiles descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-05 19:54:22,535 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 319680d28f8da52640b97eff4eaa9239 2023-03-05 19:54:22,537 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testScannerOnBulkLoadRefHFiles/319680d28f8da52640b97eff4eaa9239/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-05 19:54:22,537 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 319680d28f8da52640b97eff4eaa9239; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68713724, jitterRate=0.02391427755355835}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-05 19:54:22,537 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 319680d28f8da52640b97eff4eaa9239: 2023-03-05 19:54:22,538 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testScannerOnBulkLoadRefHFiles,1,1678046061330.319680d28f8da52640b97eff4eaa9239., pid=32, masterSystemTime=1678046062495 2023-03-05 19:54:22,540 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testScannerOnBulkLoadRefHFiles,1,1678046061330.319680d28f8da52640b97eff4eaa9239. 2023-03-05 19:54:22,540 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testScannerOnBulkLoadRefHFiles,1,1678046061330.319680d28f8da52640b97eff4eaa9239. 2023-03-05 19:54:22,540 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testScannerOnBulkLoadRefHFiles,2,1678046061330.2d86bbeed3970cf2266d199ff776cf8d. 2023-03-05 19:54:22,540 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 2d86bbeed3970cf2266d199ff776cf8d, NAME => 'testScannerOnBulkLoadRefHFiles,2,1678046061330.2d86bbeed3970cf2266d199ff776cf8d.', STARTKEY => '2', ENDKEY => ''} 2023-03-05 19:54:22,540 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testScannerOnBulkLoadRefHFiles 2d86bbeed3970cf2266d199ff776cf8d 2023-03-05 19:54:22,540 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=319680d28f8da52640b97eff4eaa9239, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:22,540 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testScannerOnBulkLoadRefHFiles,2,1678046061330.2d86bbeed3970cf2266d199ff776cf8d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-05 19:54:22,540 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 2d86bbeed3970cf2266d199ff776cf8d 2023-03-05 19:54:22,540 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testScannerOnBulkLoadRefHFiles,1,1678046061330.319680d28f8da52640b97eff4eaa9239.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1678046062540"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678046062540"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678046062540"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678046062540"}]},"ts":"1678046062540"} 2023-03-05 19:54:22,540 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 2d86bbeed3970cf2266d199ff776cf8d 2023-03-05 19:54:22,541 INFO [StoreOpener-2d86bbeed3970cf2266d199ff776cf8d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 2d86bbeed3970cf2266d199ff776cf8d 2023-03-05 19:54:22,543 DEBUG [StoreOpener-2d86bbeed3970cf2266d199ff776cf8d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testScannerOnBulkLoadRefHFiles/2d86bbeed3970cf2266d199ff776cf8d/family1 2023-03-05 19:54:22,543 DEBUG [StoreOpener-2d86bbeed3970cf2266d199ff776cf8d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testScannerOnBulkLoadRefHFiles/2d86bbeed3970cf2266d199ff776cf8d/family1 2023-03-05 19:54:22,543 INFO [StoreOpener-2d86bbeed3970cf2266d199ff776cf8d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2d86bbeed3970cf2266d199ff776cf8d columnFamilyName family1 2023-03-05 19:54:22,544 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=32, resume processing ppid=29 2023-03-05 19:54:22,544 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=32, ppid=29, state=SUCCESS; OpenRegionProcedure 319680d28f8da52640b97eff4eaa9239, server=jenkins-hbase4.apache.org,37037,1678046018983 in 198 msec 2023-03-05 19:54:22,544 INFO [StoreOpener-2d86bbeed3970cf2266d199ff776cf8d-1] regionserver.HStore(310): Store=2d86bbeed3970cf2266d199ff776cf8d/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-05 19:54:22,545 INFO [StoreOpener-2d86bbeed3970cf2266d199ff776cf8d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 2d86bbeed3970cf2266d199ff776cf8d 2023-03-05 19:54:22,545 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=29, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=319680d28f8da52640b97eff4eaa9239, ASSIGN in 358 msec 2023-03-05 19:54:22,546 DEBUG [StoreOpener-2d86bbeed3970cf2266d199ff776cf8d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testScannerOnBulkLoadRefHFiles/2d86bbeed3970cf2266d199ff776cf8d/family2 2023-03-05 19:54:22,546 DEBUG [StoreOpener-2d86bbeed3970cf2266d199ff776cf8d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testScannerOnBulkLoadRefHFiles/2d86bbeed3970cf2266d199ff776cf8d/family2 2023-03-05 19:54:22,546 INFO [StoreOpener-2d86bbeed3970cf2266d199ff776cf8d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2d86bbeed3970cf2266d199ff776cf8d columnFamilyName family2 2023-03-05 19:54:22,547 INFO [StoreOpener-2d86bbeed3970cf2266d199ff776cf8d-1] regionserver.HStore(310): Store=2d86bbeed3970cf2266d199ff776cf8d/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-05 19:54:22,548 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testScannerOnBulkLoadRefHFiles/2d86bbeed3970cf2266d199ff776cf8d 2023-03-05 19:54:22,549 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testScannerOnBulkLoadRefHFiles/2d86bbeed3970cf2266d199ff776cf8d 2023-03-05 19:54:22,550 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testScannerOnBulkLoadRefHFiles descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-05 19:54:22,551 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 2d86bbeed3970cf2266d199ff776cf8d 2023-03-05 19:54:22,553 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testScannerOnBulkLoadRefHFiles/2d86bbeed3970cf2266d199ff776cf8d/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-05 19:54:22,554 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 2d86bbeed3970cf2266d199ff776cf8d; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61709974, jitterRate=-0.08044973015785217}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-05 19:54:22,554 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 2d86bbeed3970cf2266d199ff776cf8d: 2023-03-05 19:54:22,554 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testScannerOnBulkLoadRefHFiles,2,1678046061330.2d86bbeed3970cf2266d199ff776cf8d., pid=31, masterSystemTime=1678046062495 2023-03-05 19:54:22,556 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testScannerOnBulkLoadRefHFiles,2,1678046061330.2d86bbeed3970cf2266d199ff776cf8d. 2023-03-05 19:54:22,556 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testScannerOnBulkLoadRefHFiles,2,1678046061330.2d86bbeed3970cf2266d199ff776cf8d. 2023-03-05 19:54:22,556 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=30 updating hbase:meta row=2d86bbeed3970cf2266d199ff776cf8d, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:22,557 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testScannerOnBulkLoadRefHFiles,2,1678046061330.2d86bbeed3970cf2266d199ff776cf8d.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1678046062556"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678046062556"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678046062556"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678046062556"}]},"ts":"1678046062556"} 2023-03-05 19:54:22,559 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=31, resume processing ppid=30 2023-03-05 19:54:22,560 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=31, ppid=30, state=SUCCESS; OpenRegionProcedure 2d86bbeed3970cf2266d199ff776cf8d, server=jenkins-hbase4.apache.org,37037,1678046018983 in 215 msec 2023-03-05 19:54:22,561 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=30, resume processing ppid=27 2023-03-05 19:54:22,561 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=30, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=2d86bbeed3970cf2266d199ff776cf8d, ASSIGN in 374 msec 2023-03-05 19:54:22,562 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-05 19:54:22,562 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testScannerOnBulkLoadRefHFiles","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678046062562"}]},"ts":"1678046062562"} 2023-03-05 19:54:22,563 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testScannerOnBulkLoadRefHFiles, state=ENABLED in hbase:meta 2023-03-05 19:54:22,566 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_POST_OPERATION 2023-03-05 19:54:22,568 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=27, state=SUCCESS; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles in 1.2360 sec 2023-03-05 19:54:23,167 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testScannerOnBulkLoadRefHFiles' 2023-03-05 19:54:23,442 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-03-05 19:54:23,442 INFO [Listener at localhost/40787] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testScannerOnBulkLoadRefHFiles, procId: 27 completed 2023-03-05 19:54:23,450 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-05 19:54:23,455 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(165): Submitting procedure testScannerOnBulkLoadRefHFiles 2023-03-05 19:54:23,455 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testScannerOnBulkLoadRefHFiles' 2023-03-05 19:54:23,455 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:54:23,455 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testScannerOnBulkLoadRefHFiles' starting 'acquire' 2023-03-05 19:54:23,455 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testScannerOnBulkLoadRefHFiles', kicking off acquire phase on members. 2023-03-05 19:54:23,456 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-05 19:54:23,456 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-05 19:54:23,457 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:54:23,457 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:54:23,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:54:23,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-05 19:54:23,460 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-05 19:54:23,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-05 19:54:23,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-05 19:54:23,461 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testScannerOnBulkLoadRefHFiles 2023-03-05 19:54:23,462 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:23,462 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testScannerOnBulkLoadRefHFiles 2023-03-05 19:54:23,462 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testScannerOnBulkLoadRefHFiles' with timeout 60000ms 2023-03-05 19:54:23,462 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:54:23,463 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testScannerOnBulkLoadRefHFiles' starting 'acquire' stage 2023-03-05 19:54:23,463 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:23,463 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-05 19:54:23,468 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,2,1678046061330.2d86bbeed3970cf2266d199ff776cf8d. 2023-03-05 19:54:23,468 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,2,1678046061330.2d86bbeed3970cf2266d199ff776cf8d. started... 2023-03-05 19:54:23,471 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 2d86bbeed3970cf2266d199ff776cf8d 2/2 column families, dataSize=48 B heapSize=632 B 2023-03-05 19:54:23,472 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,1,1678046061330.319680d28f8da52640b97eff4eaa9239. 2023-03-05 19:54:23,472 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,1,1678046061330.319680d28f8da52640b97eff4eaa9239. started... 2023-03-05 19:54:23,472 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 319680d28f8da52640b97eff4eaa9239: 2023-03-05 19:54:23,472 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,1,1678046061330.319680d28f8da52640b97eff4eaa9239. 2023-03-05 19:54:23,476 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-05 19:54:23,476 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-05 19:54:23,476 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-05 19:54:23,476 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,,1678046061330.237c9aa4b41fcfa45032528ec18afe9b. 2023-03-05 19:54:23,476 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,,1678046061330.237c9aa4b41fcfa45032528ec18afe9b. started... 2023-03-05 19:54:23,477 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 237c9aa4b41fcfa45032528ec18afe9b: 2023-03-05 19:54:23,477 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,,1678046061330.237c9aa4b41fcfa45032528ec18afe9b. 2023-03-05 19:54:23,477 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-05 19:54:23,900 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023030552cf847c7e2f40039484d1cc4cb60486 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c2023030552cf847c7e2f40039484d1cc4cb60486 2023-03-05 19:54:23,908 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=48, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testScannerOnBulkLoadRefHFiles/2d86bbeed3970cf2266d199ff776cf8d/.tmp/family1/248800174cd84ed58799beea2a044d6f 2023-03-05 19:54:23,913 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testScannerOnBulkLoadRefHFiles/2d86bbeed3970cf2266d199ff776cf8d/.tmp/family1/248800174cd84ed58799beea2a044d6f as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testScannerOnBulkLoadRefHFiles/2d86bbeed3970cf2266d199ff776cf8d/family1/248800174cd84ed58799beea2a044d6f 2023-03-05 19:54:23,924 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testScannerOnBulkLoadRefHFiles/2d86bbeed3970cf2266d199ff776cf8d/family1/248800174cd84ed58799beea2a044d6f, entries=1, sequenceid=5, filesize=4.9 K 2023-03-05 19:54:23,925 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~48 B/48, heapSize ~360 B/360, currentSize=0 B/0 for 2d86bbeed3970cf2266d199ff776cf8d in 454ms, sequenceid=5, compaction requested=false 2023-03-05 19:54:23,926 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 2d86bbeed3970cf2266d199ff776cf8d: 2023-03-05 19:54:23,926 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,2,1678046061330.2d86bbeed3970cf2266d199ff776cf8d. 2023-03-05 19:54:23,926 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-05 19:54:23,926 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-05 19:54:23,926 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:23,926 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testScannerOnBulkLoadRefHFiles' locally acquired 2023-03-05 19:54:23,926 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure (testScannerOnBulkLoadRefHFiles) in zk 2023-03-05 19:54:23,929 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:23,929 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-05 19:54:23,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:23,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:23,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:23,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:23,929 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-05 19:54:23,930 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testScannerOnBulkLoadRefHFiles' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-05 19:54:23,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:23,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-05 19:54:23,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:23,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:23,932 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure 'testScannerOnBulkLoadRefHFiles' on coordinator 2023-03-05 19:54:23,932 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@104e4ce2[Count = 0] remaining members to acquire global barrier 2023-03-05 19:54:23,932 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testScannerOnBulkLoadRefHFiles' starting 'in-barrier' execution. 2023-03-05 19:54:23,932 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-05 19:54:23,933 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-05 19:54:23,933 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-05 19:54:23,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-05 19:54:23,934 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testScannerOnBulkLoadRefHFiles' received 'reached' from coordinator. 2023-03-05 19:54:23,934 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testScannerOnBulkLoadRefHFiles' locally completed 2023-03-05 19:54:23,934 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testScannerOnBulkLoadRefHFiles' completed for member 'jenkins-hbase4.apache.org,37037,1678046018983' in zk 2023-03-05 19:54:23,934 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:23,934 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-05 19:54:23,935 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:23,935 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testScannerOnBulkLoadRefHFiles' has notified controller of completion 2023-03-05 19:54:23,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:23,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:23,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:23,935 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:54:23,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:23,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:23,937 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testScannerOnBulkLoadRefHFiles' completed. 2023-03-05 19:54:23,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-05 19:54:23,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:23,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:23,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-05 19:54:23,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:23,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testScannerOnBulkLoadRefHFiles' member 'jenkins-hbase4.apache.org,37037,1678046018983': 2023-03-05 19:54:23,939 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' released barrier for procedure'testScannerOnBulkLoadRefHFiles', counting down latch. Waiting for 0 more 2023-03-05 19:54:23,939 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testScannerOnBulkLoadRefHFiles' execution completed 2023-03-05 19:54:23,940 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-05 19:54:23,940 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-05 19:54:23,940 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testScannerOnBulkLoadRefHFiles 2023-03-05 19:54:23,940 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testScannerOnBulkLoadRefHFilesincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-05 19:54:23,943 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-05 19:54:23,943 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-05 19:54:23,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-05 19:54:23,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:23,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:23,943 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-05 19:54:23,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-05 19:54:23,943 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:54:23,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:23,943 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:54:23,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:54:23,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-05 19:54:23,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-05 19:54:23,944 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:23,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:23,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-05 19:54:23,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:23,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:23,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-05 19:54:23,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:23,956 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:23,963 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:23,963 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:54:23,963 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-05 19:54:23,964 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:54:23,964 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testScannerOnBulkLoadRefHFiles' 2023-03-05 19:54:23,964 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-05 19:54:23,964 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-05 19:54:23,964 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:23,964 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-05 19:54:23,964 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-05 19:54:23,964 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-05 19:54:23,964 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testScannerOnBulkLoadRefHFiles'' to complete. (max 300000 ms per retry) 2023-03-05 19:54:23,964 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-05 19:54:23,964 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:54:23,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:54:23,964 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:54:23,965 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:54:23,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:54:24,064 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-05 19:54:24,065 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-05 19:54:24,070 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-05 19:54:24,078 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(143): Procedure testScannerOnBulkLoadRefHFiles was in running list but was completed. Accepting new attempt. 2023-03-05 19:54:24,078 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(165): Submitting procedure testScannerOnBulkLoadRefHFiles 2023-03-05 19:54:24,078 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testScannerOnBulkLoadRefHFiles' 2023-03-05 19:54:24,078 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:54:24,078 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testScannerOnBulkLoadRefHFiles' starting 'acquire' 2023-03-05 19:54:24,078 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testScannerOnBulkLoadRefHFiles', kicking off acquire phase on members. 2023-03-05 19:54:24,078 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-05 19:54:24,078 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-05 19:54:24,080 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:54:24,080 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:24,080 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:54:24,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:54:24,080 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:24,080 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-05 19:54:24,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-05 19:54:24,081 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-05 19:54:24,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-05 19:54:24,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-05 19:54:24,081 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testScannerOnBulkLoadRefHFiles 2023-03-05 19:54:24,081 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testScannerOnBulkLoadRefHFiles is still present, removing 2023-03-05 19:54:24,081 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testScannerOnBulkLoadRefHFiles 2023-03-05 19:54:24,081 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testScannerOnBulkLoadRefHFiles' with timeout 60000ms 2023-03-05 19:54:24,082 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:54:24,082 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testScannerOnBulkLoadRefHFiles' starting 'acquire' stage 2023-03-05 19:54:24,082 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,2,1678046061330.2d86bbeed3970cf2266d199ff776cf8d. 2023-03-05 19:54:24,082 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,2,1678046061330.2d86bbeed3970cf2266d199ff776cf8d. started... 2023-03-05 19:54:24,082 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 2d86bbeed3970cf2266d199ff776cf8d 2/2 column families, dataSize=90 B heapSize=752 B 2023-03-05 19:54:24,083 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,1,1678046061330.319680d28f8da52640b97eff4eaa9239. 2023-03-05 19:54:24,083 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-05 19:54:24,083 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,,1678046061330.237c9aa4b41fcfa45032528ec18afe9b. 2023-03-05 19:54:24,083 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-05 19:54:24,083 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,1,1678046061330.319680d28f8da52640b97eff4eaa9239. started... 2023-03-05 19:54:24,083 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,,1678046061330.237c9aa4b41fcfa45032528ec18afe9b. started... 2023-03-05 19:54:24,083 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 319680d28f8da52640b97eff4eaa9239: 2023-03-05 19:54:24,083 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 237c9aa4b41fcfa45032528ec18afe9b: 2023-03-05 19:54:24,083 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,,1678046061330.237c9aa4b41fcfa45032528ec18afe9b. 2023-03-05 19:54:24,083 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,1,1678046061330.319680d28f8da52640b97eff4eaa9239. 2023-03-05 19:54:24,084 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-05 19:54:24,084 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-05 19:54:24,497 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230305c946bdccd83047c38117233ad067c34c to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c20230305c946bdccd83047c38117233ad067c34c 2023-03-05 19:54:24,902 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=90, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testScannerOnBulkLoadRefHFiles/2d86bbeed3970cf2266d199ff776cf8d/.tmp/family1/9cb3f4a3677942d1a280c95f6f4bf346 2023-03-05 19:54:24,908 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testScannerOnBulkLoadRefHFiles/2d86bbeed3970cf2266d199ff776cf8d/.tmp/family1/9cb3f4a3677942d1a280c95f6f4bf346 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testScannerOnBulkLoadRefHFiles/2d86bbeed3970cf2266d199ff776cf8d/family1/9cb3f4a3677942d1a280c95f6f4bf346 2023-03-05 19:54:24,913 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testScannerOnBulkLoadRefHFiles/2d86bbeed3970cf2266d199ff776cf8d/family1/9cb3f4a3677942d1a280c95f6f4bf346, entries=2, sequenceid=9, filesize=5.0 K 2023-03-05 19:54:24,913 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~90 B/90, heapSize ~480 B/480, currentSize=0 B/0 for 2d86bbeed3970cf2266d199ff776cf8d in 831ms, sequenceid=9, compaction requested=false 2023-03-05 19:54:24,914 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 2d86bbeed3970cf2266d199ff776cf8d: 2023-03-05 19:54:24,914 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,2,1678046061330.2d86bbeed3970cf2266d199ff776cf8d. 2023-03-05 19:54:24,914 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-05 19:54:24,914 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-05 19:54:24,914 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:24,914 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testScannerOnBulkLoadRefHFiles' locally acquired 2023-03-05 19:54:24,914 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure (testScannerOnBulkLoadRefHFiles) in zk 2023-03-05 19:54:24,917 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:24,917 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-05 19:54:24,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:24,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:24,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:24,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:24,918 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-05 19:54:24,918 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testScannerOnBulkLoadRefHFiles' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-05 19:54:24,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:24,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-05 19:54:24,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:24,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:24,919 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure 'testScannerOnBulkLoadRefHFiles' on coordinator 2023-03-05 19:54:24,919 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@614e0303[Count = 0] remaining members to acquire global barrier 2023-03-05 19:54:24,919 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testScannerOnBulkLoadRefHFiles' starting 'in-barrier' execution. 2023-03-05 19:54:24,919 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-05 19:54:24,921 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-05 19:54:24,921 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-05 19:54:24,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-05 19:54:24,921 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testScannerOnBulkLoadRefHFiles' received 'reached' from coordinator. 2023-03-05 19:54:24,921 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testScannerOnBulkLoadRefHFiles' locally completed 2023-03-05 19:54:24,921 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testScannerOnBulkLoadRefHFiles' completed for member 'jenkins-hbase4.apache.org,37037,1678046018983' in zk 2023-03-05 19:54:24,922 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:24,922 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-05 19:54:24,923 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testScannerOnBulkLoadRefHFiles' has notified controller of completion 2023-03-05 19:54:24,923 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:54:24,923 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testScannerOnBulkLoadRefHFiles' completed. 2023-03-05 19:54:24,923 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:24,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:24,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:24,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:24,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:24,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:24,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-05 19:54:24,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:24,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:24,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-05 19:54:24,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:24,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testScannerOnBulkLoadRefHFiles' member 'jenkins-hbase4.apache.org,37037,1678046018983': 2023-03-05 19:54:24,926 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' released barrier for procedure'testScannerOnBulkLoadRefHFiles', counting down latch. Waiting for 0 more 2023-03-05 19:54:24,926 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testScannerOnBulkLoadRefHFiles' execution completed 2023-03-05 19:54:24,927 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-05 19:54:24,927 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-05 19:54:24,927 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testScannerOnBulkLoadRefHFiles 2023-03-05 19:54:24,927 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testScannerOnBulkLoadRefHFilesincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-05 19:54:24,928 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-05 19:54:24,928 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-05 19:54:24,928 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-05 19:54:24,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-05 19:54:24,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:24,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:24,928 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:54:24,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-05 19:54:24,929 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:24,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:24,929 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:54:24,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:54:24,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-05 19:54:24,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-05 19:54:24,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:24,930 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:24,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-05 19:54:24,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:24,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:24,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-05 19:54:24,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:24,934 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:54:24,934 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:24,935 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testScannerOnBulkLoadRefHFiles' 2023-03-05 19:54:24,935 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-05 19:54:24,935 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-05 19:54:24,934 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:54:24,934 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:54:24,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:54:24,934 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:54:24,935 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testScannerOnBulkLoadRefHFiles'' to complete. (max 300000 ms per retry) 2023-03-05 19:54:24,935 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-05 19:54:24,935 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles because node does not exist (not an error) 2023-03-05 19:54:24,935 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-05 19:54:24,935 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:24,935 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-05 19:54:24,935 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-05 19:54:24,935 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-05 19:54:24,936 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:54:24,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:54:25,035 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-05 19:54:25,037 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-05 19:54:25,043 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=38221] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}] of the table testScannerOnBulkLoadRefHFiles 2023-03-05 19:54:25,044 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] procedure2.ProcedureExecutor(1029): Stored pid=34, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE 2023-03-05 19:54:25,045 DEBUG [PEWorker-4] locking.LockProcedure(309): LOCKED pid=34, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE 2023-03-05 19:54:25,046 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] mob.MobUtils(833): start MOB compaction of files for table='testScannerOnBulkLoadRefHFiles', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-05 19:54:25,046 INFO [PEWorker-4] procedure2.TimeoutExecutorThread(81): ADDED pid=34, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678046665046 2023-03-05 19:54:25,047 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-05 19:54:25,047 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 2 selected files, and 0 irrelevant files table 'testScannerOnBulkLoadRefHFiles' and column 'family1' 2023-03-05 19:54:25,047 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testScannerOnBulkLoadRefHFiles' column='family1' 2023-03-05 19:54:25,047 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] zookeeper.ReadOnlyZKClient(139): Connect 0x507c8885 to 127.0.0.1:62957 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-05 19:54:25,051 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@61e3a9e4, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-05 19:54:25,052 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230305 for table='testScannerOnBulkLoadRefHFiles' and column='family1' 2023-03-05 19:54:25,883 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023030581b17aafb0ff4056bf26fce5f0642092 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c2023030581b17aafb0ff4056bf26fce5f0642092 2023-03-05 19:54:25,884 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230305 table='testScannerOnBulkLoadRefHFiles' column='family1' 2023-03-05 19:54:25,897 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-05 19:54:25,898 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:43872, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-05 19:54:25,905 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-05 19:54:25,907 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:59424, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-05 19:54:25,917 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230305/family1/4cfed240a02c4f1b9c2b0ff03ad71b28 first=Optional[k0] last=Optional[k1] 2023-03-05 19:54:25,917 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testScannerOnBulkLoadRefHFiles,2,1678046061330.2d86bbeed3970cf2266d199ff776cf8d., hostname=jenkins-hbase4.apache.org,37037,1678046018983, seqNum=2 for row 2 with hfile group [{family1,hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230305/family1/4cfed240a02c4f1b9c2b0ff03ad71b28}] 2023-03-05 19:54:25,922 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(763): Validating hfile at hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230305/family1/4cfed240a02c4f1b9c2b0ff03ad71b28 for inclusion in 2d86bbeed3970cf2266d199ff776cf8d/family1 2023-03-05 19:54:25,925 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(775): HFile bounds: first=k0 last=k1 2023-03-05 19:54:25,925 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(777): Region bounds: first=2 last= 2023-03-05 19:54:25,925 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HRegion(2446): Flush status journal for 2d86bbeed3970cf2266d199ff776cf8d: 2023-03-05 19:54:25,926 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230305/family1/4cfed240a02c4f1b9c2b0ff03ad71b28 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testScannerOnBulkLoadRefHFiles__pdj5c7oe1gkdsjiquv0r1ho3egr3kfe2phl3mqh6u96c9k3sbk1h7uvjb3626pft/family1/4cfed240a02c4f1b9c2b0ff03ad71b28 2023-03-05 19:54:25,928 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testScannerOnBulkLoadRefHFiles__pdj5c7oe1gkdsjiquv0r1ho3egr3kfe2phl3mqh6u96c9k3sbk1h7uvjb3626pft/family1/4cfed240a02c4f1b9c2b0ff03ad71b28 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testScannerOnBulkLoadRefHFiles/2d86bbeed3970cf2266d199ff776cf8d/family1/5c6418ea8c8c49698f17cacfb10fe588_SeqId_12_ 2023-03-05 19:54:25,929 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(849): Loaded HFile hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testScannerOnBulkLoadRefHFiles__pdj5c7oe1gkdsjiquv0r1ho3egr3kfe2phl3mqh6u96c9k3sbk1h7uvjb3626pft/family1/4cfed240a02c4f1b9c2b0ff03ad71b28 into 2d86bbeed3970cf2266d199ff776cf8d/family1 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testScannerOnBulkLoadRefHFiles/2d86bbeed3970cf2266d199ff776cf8d/family1/5c6418ea8c8c49698f17cacfb10fe588_SeqId_12_ - updating store file list. 2023-03-05 19:54:25,933 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(882): Loaded HFile hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testScannerOnBulkLoadRefHFiles/2d86bbeed3970cf2266d199ff776cf8d/family1/5c6418ea8c8c49698f17cacfb10fe588_SeqId_12_ into 2d86bbeed3970cf2266d199ff776cf8d/family1 2023-03-05 19:54:25,933 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(855): Successfully loaded hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testScannerOnBulkLoadRefHFiles__pdj5c7oe1gkdsjiquv0r1ho3egr3kfe2phl3mqh6u96c9k3sbk1h7uvjb3626pft/family1/4cfed240a02c4f1b9c2b0ff03ad71b28 into 2d86bbeed3970cf2266d199ff776cf8d/family1 (new location: hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testScannerOnBulkLoadRefHFiles/2d86bbeed3970cf2266d199ff776cf8d/family1/5c6418ea8c8c49698f17cacfb10fe588_SeqId_12_) 2023-03-05 19:54:25,933 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testScannerOnBulkLoadRefHFiles__pdj5c7oe1gkdsjiquv0r1ho3egr3kfe2phl3mqh6u96c9k3sbk1h7uvjb3626pft/family1/4cfed240a02c4f1b9c2b0ff03ad71b28 2023-03-05 19:54:25,934 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-03-05 19:54:25,935 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 15445 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-03-05 19:54:25,935 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(1912): 2d86bbeed3970cf2266d199ff776cf8d/family1 is initiating minor compaction (all files) 2023-03-05 19:54:25,935 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@460b3065; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-05 19:54:25,935 INFO [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 2d86bbeed3970cf2266d199ff776cf8d/family1 in testScannerOnBulkLoadRefHFiles,2,1678046061330.2d86bbeed3970cf2266d199ff776cf8d. 2023-03-05 19:54:25,935 INFO [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testScannerOnBulkLoadRefHFiles/2d86bbeed3970cf2266d199ff776cf8d/family1/248800174cd84ed58799beea2a044d6f, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testScannerOnBulkLoadRefHFiles/2d86bbeed3970cf2266d199ff776cf8d/family1/9cb3f4a3677942d1a280c95f6f4bf346, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testScannerOnBulkLoadRefHFiles/2d86bbeed3970cf2266d199ff776cf8d/family1/5c6418ea8c8c49698f17cacfb10fe588_SeqId_12_] into tmpdir=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testScannerOnBulkLoadRefHFiles/2d86bbeed3970cf2266d199ff776cf8d/.tmp, totalSize=15.1 K 2023-03-05 19:54:25,936 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.Compactor(207): Compacting 248800174cd84ed58799beea2a044d6f, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678046063442 2023-03-05 19:54:25,936 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.Compactor(207): Compacting 9cb3f4a3677942d1a280c95f6f4bf346, keycount=2, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678046063442 2023-03-05 19:54:25,937 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.Compactor(207): Compacting 5c6418ea8c8c49698f17cacfb10fe588_SeqId_12_, keycount=2, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678046063442 2023-03-05 19:54:25,937 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230305 table='testScannerOnBulkLoadRefHFiles' column='family1' 2023-03-05 19:54:25,938 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-05 19:54:25,939 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c2023030552cf847c7e2f40039484d1cc4cb60486 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c2023030552cf847c7e2f40039484d1cc4cb60486 2023-03-05 19:54:25,941 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c20230305c946bdccd83047c38117233ad067c34c to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c20230305c946bdccd83047c38117233ad067c34c 2023-03-05 19:54:25,941 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testScannerOnBulkLoadRefHFiles 2 expired mob files are deleted 2023-03-05 19:54:25,941 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230305' 2023-03-05 19:54:25,941 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230305 for table='testScannerOnBulkLoadRefHFiles' and column='family1' 2023-03-05 19:54:25,941 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] compactions.PartitionedMobCompactor(354): After compaction, there are 1 mob files. table='testScannerOnBulkLoadRefHFiles' column='family1' 2023-03-05 19:54:25,941 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] mob.MobUtils(840): end MOB compaction of files for table='testScannerOnBulkLoadRefHFiles', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-05 19:54:25,942 DEBUG [PEWorker-1] locking.LockProcedure(242): UNLOCKED pid=34, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE 2023-03-05 19:54:25,944 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=34, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE in 899 msec 2023-03-05 19:54:25,948 INFO [RS:0;jenkins-hbase4:37037-shortCompactions-0] throttle.PressureAwareThroughputController(145): 2d86bbeed3970cf2266d199ff776cf8d#family1#compaction#37 average throughput is 0.06 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-05 19:54:25,972 INFO [Listener at localhost/40787] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testScannerOnBulkLoadRefHFiles Thread=248 (was 255), OpenFileDescriptor=585 (was 604), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=271 (was 277), ProcessCount=173 (was 173), AvailableMemoryMB=5023 (was 5227) 2023-03-05 19:54:25,982 INFO [Listener at localhost/40787] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyPolicy Thread=248, OpenFileDescriptor=585, MaxFileDescriptor=60000, SystemLoadAverage=271, ProcessCount=173, AvailableMemoryMB=5023 2023-03-05 19:54:25,984 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithWeeklyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-03-05 19:54:25,985 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure2.ProcedureExecutor(1029): Stored pid=35, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:25,987 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_PRE_OPERATION 2023-03-05 19:54:25,987 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithWeeklyPolicy" procId is: 35 2023-03-05 19:54:25,988 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-05 19:54:25,988 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-03-05 19:54:25,989 DEBUG [HFileArchiver-8] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/.tmp/data/default/testMajorCompactionWithWeeklyPolicy/5f33741eb2f558bc4b10a65a4789a550 2023-03-05 19:54:25,990 DEBUG [HFileArchiver-8] backup.HFileArchiver(153): Directory hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/.tmp/data/default/testMajorCompactionWithWeeklyPolicy/5f33741eb2f558bc4b10a65a4789a550 empty. 2023-03-05 19:54:25,990 DEBUG [HFileArchiver-8] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/.tmp/data/default/testMajorCompactionWithWeeklyPolicy/5f33741eb2f558bc4b10a65a4789a550 2023-03-05 19:54:25,990 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithWeeklyPolicy regions 2023-03-05 19:54:26,089 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-03-05 19:54:26,290 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-03-05 19:54:26,402 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/.tmp/data/default/testMajorCompactionWithWeeklyPolicy/.tabledesc/.tableinfo.0000000001 2023-03-05 19:54:26,404 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(7675): creating {ENCODED => 5f33741eb2f558bc4b10a65a4789a550, NAME => 'testMajorCompactionWithWeeklyPolicy,,1678046065984.5f33741eb2f558bc4b10a65a4789a550.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithWeeklyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/.tmp 2023-03-05 19:54:26,592 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-03-05 19:54:26,769 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testScannerOnBulkLoadRefHFiles/2d86bbeed3970cf2266d199ff776cf8d/.tmp/family1/e7e22910c4684a5982e9936826f7b93e as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testScannerOnBulkLoadRefHFiles/2d86bbeed3970cf2266d199ff776cf8d/family1/e7e22910c4684a5982e9936826f7b93e 2023-03-05 19:54:26,775 INFO [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 2d86bbeed3970cf2266d199ff776cf8d/family1 of 2d86bbeed3970cf2266d199ff776cf8d into e7e22910c4684a5982e9936826f7b93e(size=5.1 K), total size for store is 5.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-05 19:54:26,775 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 2d86bbeed3970cf2266d199ff776cf8d: 2023-03-05 19:54:26,775 INFO [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testScannerOnBulkLoadRefHFiles,2,1678046061330.2d86bbeed3970cf2266d199ff776cf8d., storeName=family1, fileCount=3, fileSize=15.1 K (4.9 K, 5.0 K, 5.2 K), priority=2, time=1678046065935; duration=0sec 2023-03-05 19:54:26,775 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-05 19:54:26,811 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testScannerOnBulkLoadRefHFiles/2d86bbeed3970cf2266d199ff776cf8d/family1/248800174cd84ed58799beea2a044d6f, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testScannerOnBulkLoadRefHFiles/2d86bbeed3970cf2266d199ff776cf8d/family1/9cb3f4a3677942d1a280c95f6f4bf346, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testScannerOnBulkLoadRefHFiles/2d86bbeed3970cf2266d199ff776cf8d/family1/5c6418ea8c8c49698f17cacfb10fe588_SeqId_12_] to archive 2023-03-05 19:54:26,811 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyPolicy,,1678046065984.5f33741eb2f558bc4b10a65a4789a550.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-05 19:54:26,812 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1604): Closing 5f33741eb2f558bc4b10a65a4789a550, disabling compactions & flushes 2023-03-05 19:54:26,812 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyPolicy,,1678046065984.5f33741eb2f558bc4b10a65a4789a550. 2023-03-05 19:54:26,812 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyPolicy,,1678046065984.5f33741eb2f558bc4b10a65a4789a550. 2023-03-05 19:54:26,812 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyPolicy,,1678046065984.5f33741eb2f558bc4b10a65a4789a550. after waiting 0 ms 2023-03-05 19:54:26,812 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyPolicy,,1678046065984.5f33741eb2f558bc4b10a65a4789a550. 2023-03-05 19:54:26,812 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyPolicy,,1678046065984.5f33741eb2f558bc4b10a65a4789a550. 2023-03-05 19:54:26,812 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1558): Region close journal for 5f33741eb2f558bc4b10a65a4789a550: 2023-03-05 19:54:26,812 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-05 19:54:26,814 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testScannerOnBulkLoadRefHFiles/2d86bbeed3970cf2266d199ff776cf8d/family1/248800174cd84ed58799beea2a044d6f to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testScannerOnBulkLoadRefHFiles/2d86bbeed3970cf2266d199ff776cf8d/family1/248800174cd84ed58799beea2a044d6f 2023-03-05 19:54:26,815 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_ADD_TO_META 2023-03-05 19:54:26,816 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testScannerOnBulkLoadRefHFiles/2d86bbeed3970cf2266d199ff776cf8d/family1/9cb3f4a3677942d1a280c95f6f4bf346 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testScannerOnBulkLoadRefHFiles/2d86bbeed3970cf2266d199ff776cf8d/family1/9cb3f4a3677942d1a280c95f6f4bf346 2023-03-05 19:54:26,816 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyPolicy,,1678046065984.5f33741eb2f558bc4b10a65a4789a550.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1678046066816"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678046066816"}]},"ts":"1678046066816"} 2023-03-05 19:54:26,818 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testScannerOnBulkLoadRefHFiles/2d86bbeed3970cf2266d199ff776cf8d/family1/5c6418ea8c8c49698f17cacfb10fe588_SeqId_12_ to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testScannerOnBulkLoadRefHFiles/2d86bbeed3970cf2266d199ff776cf8d/family1/5c6418ea8c8c49698f17cacfb10fe588_SeqId_12_ 2023-03-05 19:54:26,818 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-03-05 19:54:26,818 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-05 19:54:26,819 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678046066819"}]},"ts":"1678046066819"} 2023-03-05 19:54:26,820 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyPolicy, state=ENABLING in hbase:meta 2023-03-05 19:54:26,823 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyPolicy, region=5f33741eb2f558bc4b10a65a4789a550, ASSIGN}] 2023-03-05 19:54:26,824 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyPolicy, region=5f33741eb2f558bc4b10a65a4789a550, ASSIGN 2023-03-05 19:54:26,825 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyPolicy, region=5f33741eb2f558bc4b10a65a4789a550, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,37037,1678046018983; forceNewPlan=false, retain=false 2023-03-05 19:54:26,976 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=5f33741eb2f558bc4b10a65a4789a550, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:26,976 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyPolicy,,1678046065984.5f33741eb2f558bc4b10a65a4789a550.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1678046066976"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678046066976"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678046066976"}]},"ts":"1678046066976"} 2023-03-05 19:54:26,978 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=37, ppid=36, state=RUNNABLE; OpenRegionProcedure 5f33741eb2f558bc4b10a65a4789a550, server=jenkins-hbase4.apache.org,37037,1678046018983}] 2023-03-05 19:54:27,093 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-03-05 19:54:27,133 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyPolicy,,1678046065984.5f33741eb2f558bc4b10a65a4789a550. 2023-03-05 19:54:27,133 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 5f33741eb2f558bc4b10a65a4789a550, NAME => 'testMajorCompactionWithWeeklyPolicy,,1678046065984.5f33741eb2f558bc4b10a65a4789a550.', STARTKEY => '', ENDKEY => ''} 2023-03-05 19:54:27,133 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyPolicy 5f33741eb2f558bc4b10a65a4789a550 2023-03-05 19:54:27,134 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyPolicy,,1678046065984.5f33741eb2f558bc4b10a65a4789a550.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-05 19:54:27,134 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 5f33741eb2f558bc4b10a65a4789a550 2023-03-05 19:54:27,134 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 5f33741eb2f558bc4b10a65a4789a550 2023-03-05 19:54:27,135 INFO [StoreOpener-5f33741eb2f558bc4b10a65a4789a550-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 5f33741eb2f558bc4b10a65a4789a550 2023-03-05 19:54:27,136 DEBUG [StoreOpener-5f33741eb2f558bc4b10a65a4789a550-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyPolicy/5f33741eb2f558bc4b10a65a4789a550/family1 2023-03-05 19:54:27,136 DEBUG [StoreOpener-5f33741eb2f558bc4b10a65a4789a550-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyPolicy/5f33741eb2f558bc4b10a65a4789a550/family1 2023-03-05 19:54:27,137 INFO [StoreOpener-5f33741eb2f558bc4b10a65a4789a550-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 5f33741eb2f558bc4b10a65a4789a550 columnFamilyName family1 2023-03-05 19:54:27,138 INFO [StoreOpener-5f33741eb2f558bc4b10a65a4789a550-1] regionserver.HStore(310): Store=5f33741eb2f558bc4b10a65a4789a550/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-05 19:54:27,139 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyPolicy/5f33741eb2f558bc4b10a65a4789a550 2023-03-05 19:54:27,139 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyPolicy/5f33741eb2f558bc4b10a65a4789a550 2023-03-05 19:54:27,142 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 5f33741eb2f558bc4b10a65a4789a550 2023-03-05 19:54:27,144 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyPolicy/5f33741eb2f558bc4b10a65a4789a550/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-05 19:54:27,144 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 5f33741eb2f558bc4b10a65a4789a550; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62762666, jitterRate=-0.0647633969783783}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-05 19:54:27,144 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 5f33741eb2f558bc4b10a65a4789a550: 2023-03-05 19:54:27,145 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyPolicy,,1678046065984.5f33741eb2f558bc4b10a65a4789a550., pid=37, masterSystemTime=1678046067130 2023-03-05 19:54:27,146 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyPolicy,,1678046065984.5f33741eb2f558bc4b10a65a4789a550. 2023-03-05 19:54:27,147 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyPolicy,,1678046065984.5f33741eb2f558bc4b10a65a4789a550. 2023-03-05 19:54:27,147 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=5f33741eb2f558bc4b10a65a4789a550, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:27,147 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyPolicy,,1678046065984.5f33741eb2f558bc4b10a65a4789a550.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1678046067147"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678046067147"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678046067147"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678046067147"}]},"ts":"1678046067147"} 2023-03-05 19:54:27,151 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=37, resume processing ppid=36 2023-03-05 19:54:27,151 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=37, ppid=36, state=SUCCESS; OpenRegionProcedure 5f33741eb2f558bc4b10a65a4789a550, server=jenkins-hbase4.apache.org,37037,1678046018983 in 171 msec 2023-03-05 19:54:27,153 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=36, resume processing ppid=35 2023-03-05 19:54:27,153 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=36, ppid=35, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyPolicy, region=5f33741eb2f558bc4b10a65a4789a550, ASSIGN in 328 msec 2023-03-05 19:54:27,153 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-05 19:54:27,154 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678046067153"}]},"ts":"1678046067153"} 2023-03-05 19:54:27,160 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyPolicy, state=ENABLED in hbase:meta 2023-03-05 19:54:27,163 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_POST_OPERATION 2023-03-05 19:54:27,165 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=35, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy in 1.1790 sec 2023-03-05 19:54:28,044 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-05 19:54:28,094 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-03-05 19:54:28,095 INFO [Listener at localhost/40787] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithWeeklyPolicy, procId: 35 completed 2023-03-05 19:54:28,131 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-05 19:54:28,138 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:28,138 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-05 19:54:28,138 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:54:28,138 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-05 19:54:28,138 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-05 19:54:28,139 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:28,139 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:28,140 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:28,140 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:54:28,140 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:54:28,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:54:28,141 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:28,141 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-05 19:54:28,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:28,141 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:28,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-05 19:54:28,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:28,142 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:28,144 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:28,144 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-05 19:54:28,144 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:54:28,144 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-05 19:54:28,144 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-05 19:54:28,144 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-05 19:54:28,144 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1678046065984.5f33741eb2f558bc4b10a65a4789a550. 2023-03-05 19:54:28,144 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1678046065984.5f33741eb2f558bc4b10a65a4789a550. started... 2023-03-05 19:54:28,145 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 5f33741eb2f558bc4b10a65a4789a550 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-03-05 19:54:28,589 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130aeb85aedfa8c4ad8a716cb02cfcd9958 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151130aeb85aedfa8c4ad8a716cb02cfcd9958 2023-03-05 19:54:28,994 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyPolicy/5f33741eb2f558bc4b10a65a4789a550/.tmp/family1/eacfc7de27e843fb82f526dd3186b300 2023-03-05 19:54:29,004 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyPolicy/5f33741eb2f558bc4b10a65a4789a550/.tmp/family1/eacfc7de27e843fb82f526dd3186b300 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyPolicy/5f33741eb2f558bc4b10a65a4789a550/family1/eacfc7de27e843fb82f526dd3186b300 2023-03-05 19:54:29,010 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyPolicy/5f33741eb2f558bc4b10a65a4789a550/family1/eacfc7de27e843fb82f526dd3186b300, entries=1000, sequenceid=5, filesize=165.1 K 2023-03-05 19:54:29,012 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 5f33741eb2f558bc4b10a65a4789a550 in 867ms, sequenceid=5, compaction requested=false 2023-03-05 19:54:29,012 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyPolicy' 2023-03-05 19:54:29,014 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 5f33741eb2f558bc4b10a65a4789a550: 2023-03-05 19:54:29,014 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1678046065984.5f33741eb2f558bc4b10a65a4789a550. 2023-03-05 19:54:29,014 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-05 19:54:29,014 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-05 19:54:29,014 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:29,014 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-05 19:54:29,014 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-05 19:54:29,016 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:29,016 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:29,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:29,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:29,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:29,017 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:29,017 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-05 19:54:29,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:29,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:29,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:29,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:29,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:29,019 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-05 19:54:29,019 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3ee633ae[Count = 0] remaining members to acquire global barrier 2023-03-05 19:54:29,019 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-05 19:54:29,019 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:29,021 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:29,021 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:29,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:29,021 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-05 19:54:29,021 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-05 19:54:29,021 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,37037,1678046018983' in zk 2023-03-05 19:54:29,021 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:29,021 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-05 19:54:29,023 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:29,023 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-05 19:54:29,023 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:54:29,023 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-05 19:54:29,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:29,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:29,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:29,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:29,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:29,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:29,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:29,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:29,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:29,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:29,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,37037,1678046018983': 2023-03-05 19:54:29,027 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-05 19:54:29,027 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-05 19:54:29,027 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-05 19:54:29,027 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-05 19:54:29,027 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:29,027 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-05 19:54:29,029 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:29,029 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:29,029 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:54:29,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:29,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:29,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:29,030 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:29,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:29,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:29,030 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:54:29,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:54:29,030 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:29,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:29,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:29,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:29,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:29,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:29,032 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:29,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:29,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:29,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:29,034 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:29,034 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:29,035 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:29,035 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-05 19:54:29,035 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-05 19:54:29,035 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:54:29,035 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:29,035 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:54:29,035 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:54:29,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:54:29,035 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:29,035 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:54:29,035 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:29,035 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:29,035 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-05 19:54:29,035 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-05 19:54:29,035 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:54:29,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:54:29,135 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-05 19:54:29,136 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-05 19:54:29,140 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-05 19:54:29,141 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-05 19:54:29,142 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:29,142 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-05 19:54:29,142 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:54:29,142 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-05 19:54:29,142 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-05 19:54:29,142 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:29,142 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:29,143 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:54:29,143 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:29,143 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:54:29,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:54:29,144 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:29,144 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-05 19:54:29,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:29,144 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:29,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-05 19:54:29,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:29,145 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:29,145 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-03-05 19:54:29,145 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:29,145 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-05 19:54:29,145 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:54:29,145 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-05 19:54:29,145 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-05 19:54:29,145 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-05 19:54:29,145 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1678046065984.5f33741eb2f558bc4b10a65a4789a550. 2023-03-05 19:54:29,145 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1678046065984.5f33741eb2f558bc4b10a65a4789a550. started... 2023-03-05 19:54:29,145 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 5f33741eb2f558bc4b10a65a4789a550 1/1 column families, dataSize=74 B heapSize=408 B 2023-03-05 19:54:29,558 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015112880e83d1137f049cea974f50979db0bbc to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015112880e83d1137f049cea974f50979db0bbc 2023-03-05 19:54:29,963 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyPolicy/5f33741eb2f558bc4b10a65a4789a550/.tmp/family1/500542d9e3f64cb6b2aa5168f4363dcb 2023-03-05 19:54:29,969 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyPolicy/5f33741eb2f558bc4b10a65a4789a550/.tmp/family1/500542d9e3f64cb6b2aa5168f4363dcb as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyPolicy/5f33741eb2f558bc4b10a65a4789a550/family1/500542d9e3f64cb6b2aa5168f4363dcb 2023-03-05 19:54:29,973 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyPolicy/5f33741eb2f558bc4b10a65a4789a550/family1/500542d9e3f64cb6b2aa5168f4363dcb, entries=1, sequenceid=9, filesize=5.0 K 2023-03-05 19:54:29,974 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 5f33741eb2f558bc4b10a65a4789a550 in 829ms, sequenceid=9, compaction requested=false 2023-03-05 19:54:29,974 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 5f33741eb2f558bc4b10a65a4789a550: 2023-03-05 19:54:29,974 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1678046065984.5f33741eb2f558bc4b10a65a4789a550. 2023-03-05 19:54:29,974 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-05 19:54:29,974 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-05 19:54:29,974 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:29,974 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-05 19:54:29,974 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-05 19:54:29,976 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:29,976 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:29,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:29,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:29,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:29,976 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:29,976 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-05 19:54:29,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:29,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:29,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:29,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:29,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:29,977 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-05 19:54:29,977 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-05 19:54:29,977 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@167bf6f6[Count = 0] remaining members to acquire global barrier 2023-03-05 19:54:29,978 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:29,981 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:29,981 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:29,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:29,981 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-05 19:54:29,981 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-05 19:54:29,981 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,37037,1678046018983' in zk 2023-03-05 19:54:29,981 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:29,981 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-05 19:54:29,982 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:29,983 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-05 19:54:29,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:29,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:29,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:29,983 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:54:29,983 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-05 19:54:29,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:29,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:29,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:29,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:29,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:29,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:29,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:29,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,37037,1678046018983': 2023-03-05 19:54:29,985 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-05 19:54:29,985 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-05 19:54:29,985 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-05 19:54:29,985 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-05 19:54:29,985 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:29,985 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-05 19:54:29,987 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:29,987 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:29,987 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:29,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:29,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:29,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:29,987 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:54:29,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:29,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:29,987 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:54:29,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:54:29,987 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:29,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:29,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:29,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:29,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:29,988 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:29,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:29,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:29,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:29,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:29,991 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:29,992 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:29,992 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:29,992 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:29,992 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:54:29,992 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:54:29,992 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:54:29,992 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:54:29,992 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-05 19:54:29,992 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-05 19:54:29,992 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:29,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:54:29,992 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-05 19:54:29,992 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:29,992 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:29,992 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-05 19:54:29,992 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:54:29,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:54:29,992 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-05 19:54:30,092 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-05 19:54:30,093 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-05 19:54:30,098 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-05 19:54:30,100 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-05 19:54:30,100 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:30,100 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-05 19:54:30,100 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:54:30,100 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-05 19:54:30,100 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-05 19:54:30,100 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:30,100 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:30,102 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:54:30,102 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:30,102 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:54:30,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:54:30,102 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:30,102 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-05 19:54:30,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:30,103 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:30,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-05 19:54:30,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:30,103 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:30,103 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-03-05 19:54:30,103 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:30,103 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-05 19:54:30,103 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:54:30,103 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-05 19:54:30,104 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-05 19:54:30,104 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-05 19:54:30,104 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1678046065984.5f33741eb2f558bc4b10a65a4789a550. 2023-03-05 19:54:30,104 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1678046065984.5f33741eb2f558bc4b10a65a4789a550. started... 2023-03-05 19:54:30,104 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 5f33741eb2f558bc4b10a65a4789a550 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-05 19:54:30,517 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151201120efa58939c4f118187f8f78d6bc77c to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151201120efa58939c4f118187f8f78d6bc77c 2023-03-05 19:54:30,923 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyPolicy/5f33741eb2f558bc4b10a65a4789a550/.tmp/family1/0bd46e31640b4fec98041cc3bece9b20 2023-03-05 19:54:30,929 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyPolicy/5f33741eb2f558bc4b10a65a4789a550/.tmp/family1/0bd46e31640b4fec98041cc3bece9b20 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyPolicy/5f33741eb2f558bc4b10a65a4789a550/family1/0bd46e31640b4fec98041cc3bece9b20 2023-03-05 19:54:30,934 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyPolicy/5f33741eb2f558bc4b10a65a4789a550/family1/0bd46e31640b4fec98041cc3bece9b20, entries=1, sequenceid=13, filesize=4.9 K 2023-03-05 19:54:30,935 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 5f33741eb2f558bc4b10a65a4789a550 in 831ms, sequenceid=13, compaction requested=true 2023-03-05 19:54:30,935 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 5f33741eb2f558bc4b10a65a4789a550: 2023-03-05 19:54:30,935 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1678046065984.5f33741eb2f558bc4b10a65a4789a550. 2023-03-05 19:54:30,935 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-05 19:54:30,935 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-05 19:54:30,935 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:30,936 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-05 19:54:30,936 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-05 19:54:30,937 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:30,937 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:30,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:30,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:30,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:30,938 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:30,938 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-05 19:54:30,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:30,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:30,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:30,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:30,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:30,939 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-05 19:54:30,939 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@74309187[Count = 0] remaining members to acquire global barrier 2023-03-05 19:54:30,939 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-05 19:54:30,939 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:30,941 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:30,941 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:30,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:30,941 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-05 19:54:30,941 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-05 19:54:30,941 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,37037,1678046018983' in zk 2023-03-05 19:54:30,941 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:30,941 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-05 19:54:30,942 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-05 19:54:30,942 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:54:30,942 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:30,942 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-05 19:54:30,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:30,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:30,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:30,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:30,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:30,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:30,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:30,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:30,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:30,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:30,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,37037,1678046018983': 2023-03-05 19:54:30,945 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-05 19:54:30,945 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-05 19:54:30,945 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-05 19:54:30,945 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-05 19:54:30,945 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:30,945 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-05 19:54:30,948 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:30,948 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:30,948 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:30,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:30,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:30,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:30,948 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:54:30,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:30,949 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:54:30,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:30,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:54:30,949 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:30,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:30,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:30,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:30,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:30,950 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:30,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:30,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:30,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:30,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:30,952 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:30,952 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:30,952 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:30,952 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:54:30,952 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-05 19:54:30,953 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-05 19:54:30,952 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:30,953 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:30,953 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:30,953 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:30,953 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-05 19:54:30,953 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-05 19:54:30,953 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-05 19:54:30,954 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:54:30,954 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:54:30,954 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:54:30,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:54:30,954 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:54:30,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:54:31,053 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-05 19:54:31,054 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-05 19:54:31,058 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-05 19:54:31,060 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-05 19:54:31,060 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:31,060 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-05 19:54:31,060 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:54:31,060 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-05 19:54:31,060 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-05 19:54:31,060 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:31,060 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:31,062 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:54:31,062 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:31,062 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:54:31,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:54:31,062 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:31,062 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-05 19:54:31,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:31,062 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:31,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-05 19:54:31,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:31,063 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:31,063 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-03-05 19:54:31,063 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:31,063 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-05 19:54:31,063 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:54:31,063 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-05 19:54:31,063 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-05 19:54:31,063 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-05 19:54:31,063 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1678046065984.5f33741eb2f558bc4b10a65a4789a550. 2023-03-05 19:54:31,063 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1678046065984.5f33741eb2f558bc4b10a65a4789a550. started... 2023-03-05 19:54:31,063 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 5f33741eb2f558bc4b10a65a4789a550 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-05 19:54:31,479 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205c61d14f85d374f7da0f52349b6c73bd8 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151205c61d14f85d374f7da0f52349b6c73bd8 2023-03-05 19:54:31,885 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyPolicy/5f33741eb2f558bc4b10a65a4789a550/.tmp/family1/a8e3348192764fe88e67828bdf6841b8 2023-03-05 19:54:31,890 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyPolicy/5f33741eb2f558bc4b10a65a4789a550/.tmp/family1/a8e3348192764fe88e67828bdf6841b8 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyPolicy/5f33741eb2f558bc4b10a65a4789a550/family1/a8e3348192764fe88e67828bdf6841b8 2023-03-05 19:54:31,894 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyPolicy/5f33741eb2f558bc4b10a65a4789a550/family1/a8e3348192764fe88e67828bdf6841b8, entries=1, sequenceid=17, filesize=4.9 K 2023-03-05 19:54:31,895 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 5f33741eb2f558bc4b10a65a4789a550 in 832ms, sequenceid=17, compaction requested=true 2023-03-05 19:54:31,895 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 5f33741eb2f558bc4b10a65a4789a550: 2023-03-05 19:54:31,895 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1678046065984.5f33741eb2f558bc4b10a65a4789a550. 2023-03-05 19:54:31,895 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-05 19:54:31,895 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-05 19:54:31,895 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:31,895 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-05 19:54:31,895 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-05 19:54:31,897 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:31,897 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:31,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:31,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:31,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:31,898 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:31,898 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-05 19:54:31,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:31,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:31,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:31,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:31,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:31,899 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-05 19:54:31,899 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@65f7c219[Count = 0] remaining members to acquire global barrier 2023-03-05 19:54:31,899 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-05 19:54:31,899 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:31,902 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:31,902 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:31,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:31,902 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-05 19:54:31,902 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:31,902 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-05 19:54:31,902 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-05 19:54:31,902 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,37037,1678046018983' in zk 2023-03-05 19:54:31,903 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-05 19:54:31,903 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:31,903 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:54:31,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:31,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:31,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:31,903 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-05 19:54:31,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:31,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:31,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:31,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:31,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:31,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:31,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:31,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,37037,1678046018983': 2023-03-05 19:54:31,906 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-05 19:54:31,906 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-05 19:54:31,906 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-05 19:54:31,906 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-05 19:54:31,906 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:31,906 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-05 19:54:31,907 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:31,907 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:31,907 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:31,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:31,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:31,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:31,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:31,907 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:54:31,908 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:54:31,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:31,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:54:31,908 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:31,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:31,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:31,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:31,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:31,909 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:31,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:31,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:31,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:31,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:31,911 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:31,911 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:31,911 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:31,911 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:54:31,911 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-05 19:54:31,911 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:31,911 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-05 19:54:31,912 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-05 19:54:31,912 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:31,912 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:54:31,912 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:31,912 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:54:31,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:54:31,912 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:31,912 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-05 19:54:31,912 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:54:31,912 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-05 19:54:31,912 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:54:31,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:54:32,012 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-05 19:54:32,013 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-05 19:54:32,017 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-05 19:54:32,019 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-05 19:54:32,019 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:32,019 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-05 19:54:32,019 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:54:32,019 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-05 19:54:32,019 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-05 19:54:32,019 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:32,019 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:32,021 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:54:32,021 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:32,021 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:54:32,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:54:32,022 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:32,022 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-05 19:54:32,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:32,022 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:32,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-05 19:54:32,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:32,022 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:32,023 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-03-05 19:54:32,023 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:32,023 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-05 19:54:32,023 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:54:32,023 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-05 19:54:32,023 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-05 19:54:32,023 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-05 19:54:32,023 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1678046065984.5f33741eb2f558bc4b10a65a4789a550. 2023-03-05 19:54:32,023 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1678046065984.5f33741eb2f558bc4b10a65a4789a550. started... 2023-03-05 19:54:32,023 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 5f33741eb2f558bc4b10a65a4789a550 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-05 19:54:32,435 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015122831a18e6a5e2e46b6be38e0766c762a66 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015122831a18e6a5e2e46b6be38e0766c762a66 2023-03-05 19:54:32,841 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyPolicy/5f33741eb2f558bc4b10a65a4789a550/.tmp/family1/32efe41a4d6c40a898a30b2863d1f034 2023-03-05 19:54:32,846 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyPolicy/5f33741eb2f558bc4b10a65a4789a550/.tmp/family1/32efe41a4d6c40a898a30b2863d1f034 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyPolicy/5f33741eb2f558bc4b10a65a4789a550/family1/32efe41a4d6c40a898a30b2863d1f034 2023-03-05 19:54:32,850 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyPolicy/5f33741eb2f558bc4b10a65a4789a550/family1/32efe41a4d6c40a898a30b2863d1f034, entries=1, sequenceid=21, filesize=4.9 K 2023-03-05 19:54:32,851 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 5f33741eb2f558bc4b10a65a4789a550 in 828ms, sequenceid=21, compaction requested=true 2023-03-05 19:54:32,851 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 5f33741eb2f558bc4b10a65a4789a550: 2023-03-05 19:54:32,851 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1678046065984.5f33741eb2f558bc4b10a65a4789a550. 2023-03-05 19:54:32,851 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-05 19:54:32,851 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-05 19:54:32,851 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:32,851 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-05 19:54:32,851 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-05 19:54:32,855 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:32,855 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:32,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:32,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:32,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:32,855 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:32,855 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-05 19:54:32,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:32,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:32,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:32,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:32,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:32,857 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-05 19:54:32,857 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6a83f64a[Count = 0] remaining members to acquire global barrier 2023-03-05 19:54:32,857 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-05 19:54:32,857 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:32,858 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:32,858 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:32,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:32,858 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-05 19:54:32,858 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-05 19:54:32,858 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,37037,1678046018983' in zk 2023-03-05 19:54:32,858 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:32,858 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-05 19:54:32,861 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-05 19:54:32,861 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:54:32,861 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:32,861 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-05 19:54:32,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:32,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:32,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:32,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:32,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:32,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:32,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:32,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:32,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:32,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:32,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,37037,1678046018983': 2023-03-05 19:54:32,863 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-05 19:54:32,863 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-05 19:54:32,863 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-05 19:54:32,863 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-05 19:54:32,863 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:32,863 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-05 19:54:32,864 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:32,864 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:32,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:32,864 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:32,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:32,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:32,864 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:54:32,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:32,865 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:32,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:32,865 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:54:32,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:54:32,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:32,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:32,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:32,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:32,866 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:32,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:32,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:32,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:32,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:32,868 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:32,868 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:54:32,868 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:32,868 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:54:32,868 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:32,868 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-05 19:54:32,868 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-05 19:54:32,868 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-05 19:54:32,868 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:54:32,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:54:32,868 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:54:32,868 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:32,868 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:32,868 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:32,868 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:32,868 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-05 19:54:32,869 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:54:32,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:54:32,869 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-05 19:54:32,969 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-05 19:54:32,970 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-05 19:54:32,973 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-05 19:54:32,975 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-05 19:54:32,975 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:32,975 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-05 19:54:32,975 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:54:32,975 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-05 19:54:32,975 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-05 19:54:32,975 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:32,976 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:32,977 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:32,977 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:54:32,977 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:54:32,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:54:32,977 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:32,977 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-05 19:54:32,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:32,977 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:32,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-05 19:54:32,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:32,978 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:32,978 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-03-05 19:54:32,978 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:32,978 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-05 19:54:32,978 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:54:32,978 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-05 19:54:32,978 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-05 19:54:32,978 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-05 19:54:32,978 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1678046065984.5f33741eb2f558bc4b10a65a4789a550. 2023-03-05 19:54:32,978 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1678046065984.5f33741eb2f558bc4b10a65a4789a550. started... 2023-03-05 19:54:32,978 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 5f33741eb2f558bc4b10a65a4789a550 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-05 19:54:33,390 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231a187fc95e2914da28e2b5f0783236d26 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151231a187fc95e2914da28e2b5f0783236d26 2023-03-05 19:54:33,795 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyPolicy/5f33741eb2f558bc4b10a65a4789a550/.tmp/family1/dcf649aa62644316a5018c15d4128438 2023-03-05 19:54:33,800 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyPolicy/5f33741eb2f558bc4b10a65a4789a550/.tmp/family1/dcf649aa62644316a5018c15d4128438 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyPolicy/5f33741eb2f558bc4b10a65a4789a550/family1/dcf649aa62644316a5018c15d4128438 2023-03-05 19:54:33,803 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyPolicy/5f33741eb2f558bc4b10a65a4789a550/family1/dcf649aa62644316a5018c15d4128438, entries=1, sequenceid=25, filesize=4.9 K 2023-03-05 19:54:33,804 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 5f33741eb2f558bc4b10a65a4789a550 in 826ms, sequenceid=25, compaction requested=true 2023-03-05 19:54:33,804 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 5f33741eb2f558bc4b10a65a4789a550: 2023-03-05 19:54:33,804 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1678046065984.5f33741eb2f558bc4b10a65a4789a550. 2023-03-05 19:54:33,804 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-05 19:54:33,804 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-05 19:54:33,804 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:33,804 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-05 19:54:33,804 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-05 19:54:33,807 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:33,807 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:33,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:33,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:33,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:33,807 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:33,807 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-05 19:54:33,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:33,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:33,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:33,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:33,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:33,808 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-05 19:54:33,808 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@230671aa[Count = 0] remaining members to acquire global barrier 2023-03-05 19:54:33,808 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-05 19:54:33,809 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:33,810 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:33,810 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:33,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:33,810 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-05 19:54:33,810 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-05 19:54:33,810 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,37037,1678046018983' in zk 2023-03-05 19:54:33,810 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:33,810 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-05 19:54:33,811 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:33,811 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-05 19:54:33,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:33,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:33,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:33,811 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:54:33,812 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-05 19:54:33,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:33,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:33,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:33,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:33,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:33,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:33,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:33,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,37037,1678046018983': 2023-03-05 19:54:33,813 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-05 19:54:33,813 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-05 19:54:33,813 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-05 19:54:33,814 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-05 19:54:33,814 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:33,814 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-05 19:54:33,815 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:33,815 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:33,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:33,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:33,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:33,815 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:33,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:33,815 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:54:33,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:33,816 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:33,816 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:54:33,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:54:33,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:33,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:33,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:33,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:33,817 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:33,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:33,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:33,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:33,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:33,820 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:33,820 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:54:33,820 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:33,820 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:54:33,820 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-05 19:54:33,820 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:54:33,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:54:33,820 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-05 19:54:33,820 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-05 19:54:33,820 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:33,820 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:33,820 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:33,820 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:54:33,820 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-05 19:54:33,821 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-05 19:54:33,820 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:33,820 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:54:33,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:54:33,821 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:33,921 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-05 19:54:33,922 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-05 19:54:33,925 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-05 19:54:33,927 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-05 19:54:33,927 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:33,927 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-05 19:54:33,927 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:54:33,927 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-05 19:54:33,927 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-05 19:54:33,927 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:33,927 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:33,929 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:54:33,929 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:33,929 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:54:33,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:54:33,929 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:33,929 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-05 19:54:33,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:33,930 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:33,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-05 19:54:33,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:33,930 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:33,930 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-03-05 19:54:33,930 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:33,930 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-05 19:54:33,930 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:54:33,930 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-05 19:54:33,930 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-05 19:54:33,930 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-05 19:54:33,930 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1678046065984.5f33741eb2f558bc4b10a65a4789a550. 2023-03-05 19:54:33,930 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1678046065984.5f33741eb2f558bc4b10a65a4789a550. started... 2023-03-05 19:54:33,931 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 5f33741eb2f558bc4b10a65a4789a550 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-05 19:54:34,343 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160101a663510d71cc40ff9a3a335b48f74eb9 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20160101a663510d71cc40ff9a3a335b48f74eb9 2023-03-05 19:54:34,749 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyPolicy/5f33741eb2f558bc4b10a65a4789a550/.tmp/family1/00a517ad37e54c1184e8dedc38478fd1 2023-03-05 19:54:34,754 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyPolicy/5f33741eb2f558bc4b10a65a4789a550/.tmp/family1/00a517ad37e54c1184e8dedc38478fd1 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyPolicy/5f33741eb2f558bc4b10a65a4789a550/family1/00a517ad37e54c1184e8dedc38478fd1 2023-03-05 19:54:34,758 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyPolicy/5f33741eb2f558bc4b10a65a4789a550/family1/00a517ad37e54c1184e8dedc38478fd1, entries=1, sequenceid=29, filesize=4.9 K 2023-03-05 19:54:34,758 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 5f33741eb2f558bc4b10a65a4789a550 in 827ms, sequenceid=29, compaction requested=true 2023-03-05 19:54:34,758 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 5f33741eb2f558bc4b10a65a4789a550: 2023-03-05 19:54:34,759 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1678046065984.5f33741eb2f558bc4b10a65a4789a550. 2023-03-05 19:54:34,759 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-05 19:54:34,759 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-05 19:54:34,759 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:34,759 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-05 19:54:34,759 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-05 19:54:34,760 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:34,760 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:34,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:34,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:34,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:34,761 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:34,761 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-05 19:54:34,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:34,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:34,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:34,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:34,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:34,762 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-05 19:54:34,762 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@53ca60e4[Count = 0] remaining members to acquire global barrier 2023-03-05 19:54:34,762 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-05 19:54:34,762 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:34,763 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:34,763 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:34,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:34,763 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-05 19:54:34,763 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-05 19:54:34,763 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,37037,1678046018983' in zk 2023-03-05 19:54:34,763 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:34,763 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-05 19:54:34,764 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:34,764 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-05 19:54:34,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:34,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:34,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:34,765 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:54:34,765 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-05 19:54:34,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:34,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:34,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:34,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:34,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:34,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:34,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:34,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,37037,1678046018983': 2023-03-05 19:54:34,766 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-05 19:54:34,766 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-05 19:54:34,767 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-05 19:54:34,767 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-05 19:54:34,767 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:34,767 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-05 19:54:34,771 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:34,771 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:34,771 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:34,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:34,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:34,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:34,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:34,771 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:54:34,772 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:54:34,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:34,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:54:34,772 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:34,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:34,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:34,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:34,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:34,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:34,773 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:34,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:34,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:34,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:34,778 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:34,778 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:54:34,778 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:34,778 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:54:34,778 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-05 19:54:34,778 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-05 19:54:34,778 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:34,778 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:54:34,778 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:54:34,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:54:34,778 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:34,778 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:34,779 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:34,779 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:34,779 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-05 19:54:34,779 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-05 19:54:34,779 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:54:34,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:54:34,879 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-05 19:54:34,880 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-05 19:54:34,885 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-05 19:54:34,886 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-05 19:54:34,886 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:34,887 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-05 19:54:34,887 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:54:34,887 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-05 19:54:34,887 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-05 19:54:34,887 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:34,887 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:34,889 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:34,889 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:54:34,889 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:54:34,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:54:34,889 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:34,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:34,889 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-05 19:54:34,890 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:34,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-05 19:54:34,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:34,890 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:34,890 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-03-05 19:54:34,890 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:34,891 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-05 19:54:34,891 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:54:34,891 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-05 19:54:34,891 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-05 19:54:34,891 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-05 19:54:34,891 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1678046065984.5f33741eb2f558bc4b10a65a4789a550. 2023-03-05 19:54:34,891 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1678046065984.5f33741eb2f558bc4b10a65a4789a550. started... 2023-03-05 19:54:34,891 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 5f33741eb2f558bc4b10a65a4789a550 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-05 19:54:34,911 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601036066f2a2f4e6463088c1c214492975d2 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201601036066f2a2f4e6463088c1c214492975d2 2023-03-05 19:54:34,948 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyPolicy/5f33741eb2f558bc4b10a65a4789a550/.tmp/family1/50e8cb69d17b4f30906149893c07e013 2023-03-05 19:54:34,955 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyPolicy/5f33741eb2f558bc4b10a65a4789a550/.tmp/family1/50e8cb69d17b4f30906149893c07e013 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyPolicy/5f33741eb2f558bc4b10a65a4789a550/family1/50e8cb69d17b4f30906149893c07e013 2023-03-05 19:54:34,960 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyPolicy/5f33741eb2f558bc4b10a65a4789a550/family1/50e8cb69d17b4f30906149893c07e013, entries=1, sequenceid=33, filesize=4.9 K 2023-03-05 19:54:34,961 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 5f33741eb2f558bc4b10a65a4789a550 in 70ms, sequenceid=33, compaction requested=true 2023-03-05 19:54:34,962 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 5f33741eb2f558bc4b10a65a4789a550: 2023-03-05 19:54:34,962 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1678046065984.5f33741eb2f558bc4b10a65a4789a550. 2023-03-05 19:54:34,962 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-05 19:54:34,962 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-05 19:54:34,962 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:34,962 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-05 19:54:34,962 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-05 19:54:34,965 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:34,965 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:34,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:34,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:34,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:34,965 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:34,965 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-05 19:54:34,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:34,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:34,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:34,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:34,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:34,967 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-05 19:54:34,967 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@8f6878e[Count = 0] remaining members to acquire global barrier 2023-03-05 19:54:34,967 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-05 19:54:34,967 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:34,969 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:34,969 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:34,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:34,969 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-05 19:54:34,969 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-05 19:54:34,969 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,37037,1678046018983' in zk 2023-03-05 19:54:34,969 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:34,969 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-05 19:54:34,971 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-05 19:54:34,971 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:34,971 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:54:34,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:34,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:34,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:34,971 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-05 19:54:34,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:34,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:34,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:34,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:34,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:34,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:34,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:34,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,37037,1678046018983': 2023-03-05 19:54:34,974 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-05 19:54:34,974 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-05 19:54:34,974 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-05 19:54:34,974 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-05 19:54:34,974 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:34,974 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-05 19:54:34,976 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:34,976 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:54:34,976 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:34,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:34,976 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:34,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:34,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:34,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:34,977 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:54:34,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:54:34,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:34,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:34,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:34,978 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:34,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:34,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:34,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:34,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:34,979 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:34,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:34,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:34,983 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:34,983 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:34,983 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:34,983 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:34,983 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:34,984 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:34,984 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:34,983 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:54:34,984 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:54:34,984 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:54:34,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:54:34,984 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:54:34,984 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-05 19:54:34,984 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-05 19:54:34,984 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-05 19:54:34,985 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-05 19:54:34,987 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:54:34,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:54:35,085 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-05 19:54:35,086 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-05 19:54:35,091 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-05 19:54:35,100 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-05 19:54:35,100 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:35,100 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-05 19:54:35,100 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:54:35,100 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-05 19:54:35,100 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-05 19:54:35,101 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:35,101 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:35,103 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:35,103 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:54:35,103 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:54:35,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:54:35,103 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:35,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:35,103 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-05 19:54:35,104 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:35,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-05 19:54:35,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:35,104 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:35,104 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-03-05 19:54:35,104 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:35,104 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-05 19:54:35,105 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:54:35,105 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-05 19:54:35,107 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-05 19:54:35,107 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-05 19:54:35,107 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1678046065984.5f33741eb2f558bc4b10a65a4789a550. 2023-03-05 19:54:35,107 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1678046065984.5f33741eb2f558bc4b10a65a4789a550. started... 2023-03-05 19:54:35,107 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 5f33741eb2f558bc4b10a65a4789a550 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-05 19:54:35,525 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20150907155225793da445348605a70d7d2d58a5 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20150907155225793da445348605a70d7d2d58a5 2023-03-05 19:54:35,930 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyPolicy/5f33741eb2f558bc4b10a65a4789a550/.tmp/family1/45c5f6b555bc4a10b72e5355932ac4e8 2023-03-05 19:54:35,935 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyPolicy/5f33741eb2f558bc4b10a65a4789a550/.tmp/family1/45c5f6b555bc4a10b72e5355932ac4e8 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyPolicy/5f33741eb2f558bc4b10a65a4789a550/family1/45c5f6b555bc4a10b72e5355932ac4e8 2023-03-05 19:54:35,940 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyPolicy/5f33741eb2f558bc4b10a65a4789a550/family1/45c5f6b555bc4a10b72e5355932ac4e8, entries=1, sequenceid=37, filesize=4.9 K 2023-03-05 19:54:35,941 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 5f33741eb2f558bc4b10a65a4789a550 in 834ms, sequenceid=37, compaction requested=true 2023-03-05 19:54:35,941 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 5f33741eb2f558bc4b10a65a4789a550: 2023-03-05 19:54:35,941 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1678046065984.5f33741eb2f558bc4b10a65a4789a550. 2023-03-05 19:54:35,941 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-05 19:54:35,941 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-05 19:54:35,941 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:35,941 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-05 19:54:35,941 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-05 19:54:35,942 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:35,942 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:35,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:35,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:35,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:35,943 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:35,943 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-05 19:54:35,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:35,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:35,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:35,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:35,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:35,944 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-05 19:54:35,944 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7db2017c[Count = 0] remaining members to acquire global barrier 2023-03-05 19:54:35,944 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-05 19:54:35,944 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:35,945 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:35,945 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:35,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:35,946 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:35,946 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-05 19:54:35,946 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-05 19:54:35,946 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-05 19:54:35,946 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,37037,1678046018983' in zk 2023-03-05 19:54:35,949 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-05 19:54:35,949 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:54:35,949 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:35,949 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-05 19:54:35,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:35,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:35,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:35,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:35,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:35,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:35,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:35,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:35,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:35,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:35,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,37037,1678046018983': 2023-03-05 19:54:35,951 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-05 19:54:35,951 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-05 19:54:35,951 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-05 19:54:35,951 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-05 19:54:35,951 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:35,951 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-05 19:54:35,952 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:35,952 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:35,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:35,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:35,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:35,952 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:35,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:35,952 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:54:35,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:35,953 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:35,953 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:54:35,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:54:35,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:35,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:35,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:35,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:35,954 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:35,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:35,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:35,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:35,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:35,956 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:35,956 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:54:35,956 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:35,956 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:54:35,956 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:54:35,956 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-05 19:54:35,956 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-05 19:54:35,956 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:54:35,956 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:35,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:54:35,956 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:35,956 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:35,956 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:35,956 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-05 19:54:35,957 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-05 19:54:35,956 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:35,957 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:54:35,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:54:36,057 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-05 19:54:36,057 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-05 19:54:36,061 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-05 19:54:36,062 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-05 19:54:36,063 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:36,063 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-05 19:54:36,063 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:54:36,063 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-05 19:54:36,063 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-05 19:54:36,063 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:36,063 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:36,064 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:36,064 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:54:36,065 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:54:36,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:54:36,065 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:36,065 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-05 19:54:36,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:36,065 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:36,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-05 19:54:36,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:36,065 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:36,066 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-03-05 19:54:36,066 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:36,066 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-05 19:54:36,066 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:54:36,066 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-05 19:54:36,066 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-05 19:54:36,066 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-05 19:54:36,066 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1678046065984.5f33741eb2f558bc4b10a65a4789a550. 2023-03-05 19:54:36,066 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1678046065984.5f33741eb2f558bc4b10a65a4789a550. started... 2023-03-05 19:54:36,066 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 5f33741eb2f558bc4b10a65a4789a550 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-05 19:54:36,478 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151120b9ee576eaa5e402f8628d980fc2145c5 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151120b9ee576eaa5e402f8628d980fc2145c5 2023-03-05 19:54:36,884 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyPolicy/5f33741eb2f558bc4b10a65a4789a550/.tmp/family1/7abed33aedae4da6b3e92ec42d9dc9ae 2023-03-05 19:54:36,888 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyPolicy/5f33741eb2f558bc4b10a65a4789a550/.tmp/family1/7abed33aedae4da6b3e92ec42d9dc9ae as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyPolicy/5f33741eb2f558bc4b10a65a4789a550/family1/7abed33aedae4da6b3e92ec42d9dc9ae 2023-03-05 19:54:36,892 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyPolicy/5f33741eb2f558bc4b10a65a4789a550/family1/7abed33aedae4da6b3e92ec42d9dc9ae, entries=1, sequenceid=41, filesize=4.9 K 2023-03-05 19:54:36,893 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 5f33741eb2f558bc4b10a65a4789a550 in 827ms, sequenceid=41, compaction requested=true 2023-03-05 19:54:36,893 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 5f33741eb2f558bc4b10a65a4789a550: 2023-03-05 19:54:36,893 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1678046065984.5f33741eb2f558bc4b10a65a4789a550. 2023-03-05 19:54:36,893 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-05 19:54:36,893 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-05 19:54:36,893 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:36,893 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-05 19:54:36,893 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-05 19:54:36,897 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:36,897 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:36,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:36,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:36,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:36,897 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:36,897 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-05 19:54:36,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:36,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:36,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:36,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:36,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:36,898 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-05 19:54:36,898 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-05 19:54:36,898 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6d29b441[Count = 0] remaining members to acquire global barrier 2023-03-05 19:54:36,898 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:36,900 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:36,900 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:36,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:36,900 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-05 19:54:36,900 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-05 19:54:36,900 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,37037,1678046018983' in zk 2023-03-05 19:54:36,900 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:36,900 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-05 19:54:36,902 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:36,902 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-05 19:54:36,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:36,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:36,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:36,902 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:54:36,902 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-05 19:54:36,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:36,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:36,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:36,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:36,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:36,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:36,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:36,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,37037,1678046018983': 2023-03-05 19:54:36,904 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-05 19:54:36,904 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-05 19:54:36,904 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-05 19:54:36,904 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-05 19:54:36,904 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:36,904 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-05 19:54:36,905 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:36,905 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:36,905 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:36,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:36,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:36,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:36,905 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:54:36,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:36,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:36,906 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:36,906 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:54:36,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:54:36,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:36,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:36,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:36,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:36,907 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:36,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:36,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:36,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:36,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:36,912 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:36,913 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:36,913 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:36,913 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:36,913 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:36,913 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-05 19:54:36,913 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-05 19:54:36,913 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:54:36,913 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-05 19:54:36,913 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:36,913 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:54:36,913 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:36,913 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:54:36,913 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:54:36,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:54:36,913 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-05 19:54:36,913 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-05 19:54:36,913 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:54:36,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:54:37,013 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-05 19:54:37,014 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-05 19:54:37,016 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=38221] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyPolicy 2023-03-05 19:54:37,018 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] procedure2.ProcedureExecutor(1029): Stored pid=38, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-03-05 19:54:37,020 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=38, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-03-05 19:54:37,021 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyPolicy', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-05 19:54:37,021 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=38, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678046677021 2023-03-05 19:54:37,023 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-05 19:54:37,025 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyPolicy' and column 'family1' 2023-03-05 19:54:37,025 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-03-05 19:54:37,025 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] zookeeper.ReadOnlyZKClient(139): Connect 0x1f3acba3 to 127.0.0.1:62957 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-05 19:54:37,030 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@63eaed29, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-05 19:54:37,030 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-05 19:54:37,869 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2016010354c9355659a24e4c96ccf103d29681ea to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2016010354c9355659a24e4c96ccf103d29681ea 2023-03-05 19:54:37,871 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-03-05 19:54:37,883 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-05 19:54:37,885 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:34368, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-05 19:54:37,891 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-05 19:54:37,892 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:58240, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-05 19:54:37,900 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/3281f9f390e64d4aaca89dfbac8bd934 first=Optional[r3] last=Optional[r6] 2023-03-05 19:54:37,901 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyPolicy,,1678046065984.5f33741eb2f558bc4b10a65a4789a550., hostname=jenkins-hbase4.apache.org,37037,1678046018983, seqNum=2 for row with hfile group [{family1,hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/3281f9f390e64d4aaca89dfbac8bd934}] 2023-03-05 19:54:37,904 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(763): Validating hfile at hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/3281f9f390e64d4aaca89dfbac8bd934 for inclusion in 5f33741eb2f558bc4b10a65a4789a550/family1 2023-03-05 19:54:37,906 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-03-05 19:54:37,907 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(777): Region bounds: first= last= 2023-03-05 19:54:37,907 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HRegion(2446): Flush status journal for 5f33741eb2f558bc4b10a65a4789a550: 2023-03-05 19:54:37,907 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/3281f9f390e64d4aaca89dfbac8bd934 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testMajorCompactionWithWeeklyPolicy__jkeq2vcin6efgm1ritku686soitav7hr5r4g0h0q24vg6m9bqcj5m7o20nhmcjgc/family1/3281f9f390e64d4aaca89dfbac8bd934 2023-03-05 19:54:37,910 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testMajorCompactionWithWeeklyPolicy__jkeq2vcin6efgm1ritku686soitav7hr5r4g0h0q24vg6m9bqcj5m7o20nhmcjgc/family1/3281f9f390e64d4aaca89dfbac8bd934 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyPolicy/5f33741eb2f558bc4b10a65a4789a550/family1/fff021474ab848cba68fe15e469db743_SeqId_44_ 2023-03-05 19:54:37,910 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(849): Loaded HFile hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testMajorCompactionWithWeeklyPolicy__jkeq2vcin6efgm1ritku686soitav7hr5r4g0h0q24vg6m9bqcj5m7o20nhmcjgc/family1/3281f9f390e64d4aaca89dfbac8bd934 into 5f33741eb2f558bc4b10a65a4789a550/family1 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyPolicy/5f33741eb2f558bc4b10a65a4789a550/family1/fff021474ab848cba68fe15e469db743_SeqId_44_ - updating store file list. 2023-03-05 19:54:37,914 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(882): Loaded HFile hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyPolicy/5f33741eb2f558bc4b10a65a4789a550/family1/fff021474ab848cba68fe15e469db743_SeqId_44_ into 5f33741eb2f558bc4b10a65a4789a550/family1 2023-03-05 19:54:37,914 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(855): Successfully loaded hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testMajorCompactionWithWeeklyPolicy__jkeq2vcin6efgm1ritku686soitav7hr5r4g0h0q24vg6m9bqcj5m7o20nhmcjgc/family1/3281f9f390e64d4aaca89dfbac8bd934 into 5f33741eb2f558bc4b10a65a4789a550/family1 (new location: hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyPolicy/5f33741eb2f558bc4b10a65a4789a550/family1/fff021474ab848cba68fe15e469db743_SeqId_44_) 2023-03-05 19:54:37,914 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testMajorCompactionWithWeeklyPolicy__jkeq2vcin6efgm1ritku686soitav7hr5r4g0h0q24vg6m9bqcj5m7o20nhmcjgc/family1/3281f9f390e64d4aaca89dfbac8bd934 2023-03-05 19:54:37,914 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-03-05 19:54:37,917 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51214 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-03-05 19:54:37,917 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(1912): 5f33741eb2f558bc4b10a65a4789a550/family1 is initiating minor compaction 2023-03-05 19:54:37,917 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@6306b1c1; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-05 19:54:37,917 INFO [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 5f33741eb2f558bc4b10a65a4789a550/family1 in testMajorCompactionWithWeeklyPolicy,,1678046065984.5f33741eb2f558bc4b10a65a4789a550. 2023-03-05 19:54:37,918 INFO [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyPolicy/5f33741eb2f558bc4b10a65a4789a550/family1/500542d9e3f64cb6b2aa5168f4363dcb, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyPolicy/5f33741eb2f558bc4b10a65a4789a550/family1/0bd46e31640b4fec98041cc3bece9b20, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyPolicy/5f33741eb2f558bc4b10a65a4789a550/family1/a8e3348192764fe88e67828bdf6841b8, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyPolicy/5f33741eb2f558bc4b10a65a4789a550/family1/32efe41a4d6c40a898a30b2863d1f034, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyPolicy/5f33741eb2f558bc4b10a65a4789a550/family1/dcf649aa62644316a5018c15d4128438, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyPolicy/5f33741eb2f558bc4b10a65a4789a550/family1/00a517ad37e54c1184e8dedc38478fd1, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyPolicy/5f33741eb2f558bc4b10a65a4789a550/family1/50e8cb69d17b4f30906149893c07e013, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyPolicy/5f33741eb2f558bc4b10a65a4789a550/family1/45c5f6b555bc4a10b72e5355932ac4e8, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyPolicy/5f33741eb2f558bc4b10a65a4789a550/family1/7abed33aedae4da6b3e92ec42d9dc9ae, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyPolicy/5f33741eb2f558bc4b10a65a4789a550/family1/fff021474ab848cba68fe15e469db743_SeqId_44_] into tmpdir=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyPolicy/5f33741eb2f558bc4b10a65a4789a550/.tmp, totalSize=50.0 K 2023-03-05 19:54:37,918 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.Compactor(207): Compacting 500542d9e3f64cb6b2aa5168f4363dcb, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-03-05 19:54:37,918 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.Compactor(207): Compacting 0bd46e31640b4fec98041cc3bece9b20, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=13 2023-03-05 19:54:37,918 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.Compactor(207): Compacting a8e3348192764fe88e67828bdf6841b8, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=17 2023-03-05 19:54:37,918 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.Compactor(207): Compacting 32efe41a4d6c40a898a30b2863d1f034, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=21 2023-03-05 19:54:37,918 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.Compactor(207): Compacting dcf649aa62644316a5018c15d4128438, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=25 2023-03-05 19:54:37,918 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.Compactor(207): Compacting 00a517ad37e54c1184e8dedc38478fd1, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=29 2023-03-05 19:54:37,918 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.Compactor(207): Compacting 50e8cb69d17b4f30906149893c07e013, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=33 2023-03-05 19:54:37,918 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.Compactor(207): Compacting 45c5f6b555bc4a10b72e5355932ac4e8, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=37 2023-03-05 19:54:37,918 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.Compactor(207): Compacting 7abed33aedae4da6b3e92ec42d9dc9ae, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=41 2023-03-05 19:54:37,918 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.Compactor(207): Compacting fff021474ab848cba68fe15e469db743_SeqId_44_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=45 2023-03-05 19:54:37,919 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-03-05 19:54:37,920 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-05 19:54:37,922 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015122831a18e6a5e2e46b6be38e0766c762a66 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015122831a18e6a5e2e46b6be38e0766c762a66 2023-03-05 19:54:37,923 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151231a187fc95e2914da28e2b5f0783236d26 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151231a187fc95e2914da28e2b5f0783236d26 2023-03-05 19:54:37,924 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20160101a663510d71cc40ff9a3a335b48f74eb9 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20160101a663510d71cc40ff9a3a335b48f74eb9 2023-03-05 19:54:37,925 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201601036066f2a2f4e6463088c1c214492975d2 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201601036066f2a2f4e6463088c1c214492975d2 2023-03-05 19:54:37,925 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyPolicy 4 expired mob files are deleted 2023-03-05 19:54:37,925 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-03-05 19:54:37,926 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-05 19:54:37,926 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-05 19:54:37,926 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-05 19:54:37,926 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-05 19:54:37,926 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-05 19:54:37,926 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-05 19:54:37,926 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-05 19:54:37,926 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-05 19:54:37,953 INFO [RS:0;jenkins-hbase4:37037-shortCompactions-0] throttle.PressureAwareThroughputController(145): 5f33741eb2f558bc4b10a65a4789a550#family1#compaction#48 average throughput is 0.45 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-05 19:54:38,379 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyPolicy/5f33741eb2f558bc4b10a65a4789a550/.tmp/family1/597f5ec376bb4939a61211a45ea64e95 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyPolicy/5f33741eb2f558bc4b10a65a4789a550/family1/597f5ec376bb4939a61211a45ea64e95 2023-03-05 19:54:38,385 INFO [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 5f33741eb2f558bc4b10a65a4789a550/family1 of 5f33741eb2f558bc4b10a65a4789a550 into 597f5ec376bb4939a61211a45ea64e95(size=6.6 K), total size for store is 171.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-05 19:54:38,385 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 5f33741eb2f558bc4b10a65a4789a550: 2023-03-05 19:54:38,385 INFO [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyPolicy,,1678046065984.5f33741eb2f558bc4b10a65a4789a550., storeName=family1, fileCount=10, fileSize=50.0 K (5.0 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 5.5 K), priority=2, time=1678046077917; duration=0sec 2023-03-05 19:54:38,385 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-05 19:54:38,405 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyPolicy/5f33741eb2f558bc4b10a65a4789a550/family1/500542d9e3f64cb6b2aa5168f4363dcb, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyPolicy/5f33741eb2f558bc4b10a65a4789a550/family1/0bd46e31640b4fec98041cc3bece9b20, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyPolicy/5f33741eb2f558bc4b10a65a4789a550/family1/a8e3348192764fe88e67828bdf6841b8, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyPolicy/5f33741eb2f558bc4b10a65a4789a550/family1/32efe41a4d6c40a898a30b2863d1f034, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyPolicy/5f33741eb2f558bc4b10a65a4789a550/family1/dcf649aa62644316a5018c15d4128438, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyPolicy/5f33741eb2f558bc4b10a65a4789a550/family1/00a517ad37e54c1184e8dedc38478fd1, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyPolicy/5f33741eb2f558bc4b10a65a4789a550/family1/50e8cb69d17b4f30906149893c07e013, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyPolicy/5f33741eb2f558bc4b10a65a4789a550/family1/45c5f6b555bc4a10b72e5355932ac4e8, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyPolicy/5f33741eb2f558bc4b10a65a4789a550/family1/7abed33aedae4da6b3e92ec42d9dc9ae, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyPolicy/5f33741eb2f558bc4b10a65a4789a550/family1/fff021474ab848cba68fe15e469db743_SeqId_44_] to archive 2023-03-05 19:54:38,406 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-05 19:54:38,408 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyPolicy/5f33741eb2f558bc4b10a65a4789a550/family1/500542d9e3f64cb6b2aa5168f4363dcb to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionWithWeeklyPolicy/5f33741eb2f558bc4b10a65a4789a550/family1/500542d9e3f64cb6b2aa5168f4363dcb 2023-03-05 19:54:38,409 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyPolicy/5f33741eb2f558bc4b10a65a4789a550/family1/0bd46e31640b4fec98041cc3bece9b20 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionWithWeeklyPolicy/5f33741eb2f558bc4b10a65a4789a550/family1/0bd46e31640b4fec98041cc3bece9b20 2023-03-05 19:54:38,410 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyPolicy/5f33741eb2f558bc4b10a65a4789a550/family1/a8e3348192764fe88e67828bdf6841b8 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionWithWeeklyPolicy/5f33741eb2f558bc4b10a65a4789a550/family1/a8e3348192764fe88e67828bdf6841b8 2023-03-05 19:54:38,411 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyPolicy/5f33741eb2f558bc4b10a65a4789a550/family1/32efe41a4d6c40a898a30b2863d1f034 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionWithWeeklyPolicy/5f33741eb2f558bc4b10a65a4789a550/family1/32efe41a4d6c40a898a30b2863d1f034 2023-03-05 19:54:38,412 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyPolicy/5f33741eb2f558bc4b10a65a4789a550/family1/dcf649aa62644316a5018c15d4128438 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionWithWeeklyPolicy/5f33741eb2f558bc4b10a65a4789a550/family1/dcf649aa62644316a5018c15d4128438 2023-03-05 19:54:38,413 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyPolicy/5f33741eb2f558bc4b10a65a4789a550/family1/00a517ad37e54c1184e8dedc38478fd1 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionWithWeeklyPolicy/5f33741eb2f558bc4b10a65a4789a550/family1/00a517ad37e54c1184e8dedc38478fd1 2023-03-05 19:54:38,414 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyPolicy/5f33741eb2f558bc4b10a65a4789a550/family1/50e8cb69d17b4f30906149893c07e013 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionWithWeeklyPolicy/5f33741eb2f558bc4b10a65a4789a550/family1/50e8cb69d17b4f30906149893c07e013 2023-03-05 19:54:38,415 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyPolicy/5f33741eb2f558bc4b10a65a4789a550/family1/45c5f6b555bc4a10b72e5355932ac4e8 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionWithWeeklyPolicy/5f33741eb2f558bc4b10a65a4789a550/family1/45c5f6b555bc4a10b72e5355932ac4e8 2023-03-05 19:54:38,416 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyPolicy/5f33741eb2f558bc4b10a65a4789a550/family1/7abed33aedae4da6b3e92ec42d9dc9ae to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionWithWeeklyPolicy/5f33741eb2f558bc4b10a65a4789a550/family1/7abed33aedae4da6b3e92ec42d9dc9ae 2023-03-05 19:54:38,417 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyPolicy/5f33741eb2f558bc4b10a65a4789a550/family1/fff021474ab848cba68fe15e469db743_SeqId_44_ to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionWithWeeklyPolicy/5f33741eb2f558bc4b10a65a4789a550/family1/fff021474ab848cba68fe15e469db743_SeqId_44_ 2023-03-05 19:54:38,788 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120558e998b9d9d543bbb48ac3a9e0d073d8 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015120558e998b9d9d543bbb48ac3a9e0d073d8 2023-03-05 19:54:38,788 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-03-05 19:54:38,816 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/bb6211dba5ea4e199634e3d66516f582 first=Optional[r00] last=Optional[r2] 2023-03-05 19:54:38,817 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyPolicy,,1678046065984.5f33741eb2f558bc4b10a65a4789a550., hostname=jenkins-hbase4.apache.org,37037,1678046018983, seqNum=2 for row with hfile group [{family1,hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/bb6211dba5ea4e199634e3d66516f582}] 2023-03-05 19:54:38,820 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(763): Validating hfile at hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/bb6211dba5ea4e199634e3d66516f582 for inclusion in 5f33741eb2f558bc4b10a65a4789a550/family1 2023-03-05 19:54:38,824 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(775): HFile bounds: first=r00 last=r2 2023-03-05 19:54:38,824 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(777): Region bounds: first= last= 2023-03-05 19:54:38,824 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HRegion(2446): Flush status journal for 5f33741eb2f558bc4b10a65a4789a550: 2023-03-05 19:54:38,825 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/bb6211dba5ea4e199634e3d66516f582 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testMajorCompactionWithWeeklyPolicy__38nissfi4ccp09jeaqvufr1fc2dvo6i48ks3elu5rllor78o6t13celsnfop2td0/family1/bb6211dba5ea4e199634e3d66516f582 2023-03-05 19:54:38,827 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testMajorCompactionWithWeeklyPolicy__38nissfi4ccp09jeaqvufr1fc2dvo6i48ks3elu5rllor78o6t13celsnfop2td0/family1/bb6211dba5ea4e199634e3d66516f582 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyPolicy/5f33741eb2f558bc4b10a65a4789a550/family1/4adcfd5727cf4d62abe246c542ecff67_SeqId_47_ 2023-03-05 19:54:38,828 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(849): Loaded HFile hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testMajorCompactionWithWeeklyPolicy__38nissfi4ccp09jeaqvufr1fc2dvo6i48ks3elu5rllor78o6t13celsnfop2td0/family1/bb6211dba5ea4e199634e3d66516f582 into 5f33741eb2f558bc4b10a65a4789a550/family1 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyPolicy/5f33741eb2f558bc4b10a65a4789a550/family1/4adcfd5727cf4d62abe246c542ecff67_SeqId_47_ - updating store file list. 2023-03-05 19:54:38,832 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(882): Loaded HFile hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyPolicy/5f33741eb2f558bc4b10a65a4789a550/family1/4adcfd5727cf4d62abe246c542ecff67_SeqId_47_ into 5f33741eb2f558bc4b10a65a4789a550/family1 2023-03-05 19:54:38,832 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(855): Successfully loaded hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testMajorCompactionWithWeeklyPolicy__38nissfi4ccp09jeaqvufr1fc2dvo6i48ks3elu5rllor78o6t13celsnfop2td0/family1/bb6211dba5ea4e199634e3d66516f582 into 5f33741eb2f558bc4b10a65a4789a550/family1 (new location: hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyPolicy/5f33741eb2f558bc4b10a65a4789a550/family1/4adcfd5727cf4d62abe246c542ecff67_SeqId_47_) 2023-03-05 19:54:38,833 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testMajorCompactionWithWeeklyPolicy__38nissfi4ccp09jeaqvufr1fc2dvo6i48ks3elu5rllor78o6t13celsnfop2td0/family1/bb6211dba5ea4e199634e3d66516f582 2023-03-05 19:54:38,833 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-03-05 19:54:38,834 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 345120 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-03-05 19:54:38,834 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(1912): 5f33741eb2f558bc4b10a65a4789a550/family1 is initiating minor compaction (all files) 2023-03-05 19:54:38,834 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@1778e465; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-05 19:54:38,835 INFO [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 5f33741eb2f558bc4b10a65a4789a550/family1 in testMajorCompactionWithWeeklyPolicy,,1678046065984.5f33741eb2f558bc4b10a65a4789a550. 2023-03-05 19:54:38,835 INFO [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyPolicy/5f33741eb2f558bc4b10a65a4789a550/family1/eacfc7de27e843fb82f526dd3186b300, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyPolicy/5f33741eb2f558bc4b10a65a4789a550/family1/597f5ec376bb4939a61211a45ea64e95, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyPolicy/5f33741eb2f558bc4b10a65a4789a550/family1/4adcfd5727cf4d62abe246c542ecff67_SeqId_47_] into tmpdir=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyPolicy/5f33741eb2f558bc4b10a65a4789a550/.tmp, totalSize=337.0 K 2023-03-05 19:54:38,835 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.Compactor(207): Compacting eacfc7de27e843fb82f526dd3186b300, keycount=1000, bloomtype=ROW, size=165.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878843274 2023-03-05 19:54:38,836 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.Compactor(207): Compacting 597f5ec376bb4939a61211a45ea64e95, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621243274 2023-03-05 19:54:38,836 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.Compactor(207): Compacting 4adcfd5727cf4d62abe246c542ecff67_SeqId_47_, keycount=1002, bloomtype=ROW, size=165.4 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448878843274 2023-03-05 19:54:38,837 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-03-05 19:54:38,839 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-05 19:54:38,841 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151130aeb85aedfa8c4ad8a716cb02cfcd9958 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151130aeb85aedfa8c4ad8a716cb02cfcd9958 2023-03-05 19:54:38,842 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151201120efa58939c4f118187f8f78d6bc77c to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151201120efa58939c4f118187f8f78d6bc77c 2023-03-05 19:54:38,843 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151205c61d14f85d374f7da0f52349b6c73bd8 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151205c61d14f85d374f7da0f52349b6c73bd8 2023-03-05 19:54:38,843 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyPolicy 3 expired mob files are deleted 2023-03-05 19:54:38,843 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-03-05 19:54:38,844 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-05 19:54:38,844 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-03-05 19:54:38,844 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyPolicy', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-05 19:54:38,845 DEBUG [PEWorker-4] locking.LockProcedure(242): UNLOCKED pid=38, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-03-05 19:54:38,847 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=38, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE in 1.8280 sec 2023-03-05 19:54:38,854 INFO [RS:0;jenkins-hbase4:37037-shortCompactions-0] throttle.PressureAwareThroughputController(145): 5f33741eb2f558bc4b10a65a4789a550#family1#compaction#49 average throughput is 19.33 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-05 19:54:38,934 INFO [Listener at localhost/40787] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyPolicy Thread=256 (was 248) - Thread LEAK? -, OpenFileDescriptor=618 (was 585) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=231 (was 271), ProcessCount=173 (was 173), AvailableMemoryMB=4841 (was 5023) 2023-03-05 19:54:38,946 INFO [Listener at localhost/40787] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly Thread=256, OpenFileDescriptor=618, MaxFileDescriptor=60000, SystemLoadAverage=231, ProcessCount=173, AvailableMemoryMB=4839 2023-03-05 19:54:38,948 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-03-05 19:54:38,949 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure2.ProcedureExecutor(1029): Stored pid=39, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:38,951 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_PRE_OPERATION 2023-03-05 19:54:38,951 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly" procId is: 39 2023-03-05 19:54:38,951 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-05 19:54:38,952 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-03-05 19:54:38,953 DEBUG [HFileArchiver-9] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4 2023-03-05 19:54:38,953 DEBUG [HFileArchiver-9] backup.HFileArchiver(153): Directory hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4 empty. 2023-03-05 19:54:38,954 DEBUG [HFileArchiver-9] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4 2023-03-05 19:54:38,954 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly regions 2023-03-05 19:54:39,053 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-03-05 19:54:39,253 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-03-05 19:54:39,363 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000001 2023-03-05 19:54:39,365 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(7675): creating {ENCODED => bf39f342faf613faca7f43b94cc73bc4, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678046078948.bf39f342faf613faca7f43b94cc73bc4.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/.tmp 2023-03-05 19:54:39,370 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678046078948.bf39f342faf613faca7f43b94cc73bc4.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-05 19:54:39,370 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1604): Closing bf39f342faf613faca7f43b94cc73bc4, disabling compactions & flushes 2023-03-05 19:54:39,370 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678046078948.bf39f342faf613faca7f43b94cc73bc4. 2023-03-05 19:54:39,370 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678046078948.bf39f342faf613faca7f43b94cc73bc4. 2023-03-05 19:54:39,370 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678046078948.bf39f342faf613faca7f43b94cc73bc4. after waiting 0 ms 2023-03-05 19:54:39,370 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678046078948.bf39f342faf613faca7f43b94cc73bc4. 2023-03-05 19:54:39,370 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678046078948.bf39f342faf613faca7f43b94cc73bc4. 2023-03-05 19:54:39,370 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1558): Region close journal for bf39f342faf613faca7f43b94cc73bc4: 2023-03-05 19:54:39,372 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_ADD_TO_META 2023-03-05 19:54:39,373 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678046078948.bf39f342faf613faca7f43b94cc73bc4.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1678046079373"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678046079373"}]},"ts":"1678046079373"} 2023-03-05 19:54:39,375 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-03-05 19:54:39,375 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-05 19:54:39,375 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678046079375"}]},"ts":"1678046079375"} 2023-03-05 19:54:39,376 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, state=ENABLING in hbase:meta 2023-03-05 19:54:39,379 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=40, ppid=39, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=bf39f342faf613faca7f43b94cc73bc4, ASSIGN}] 2023-03-05 19:54:39,381 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=40, ppid=39, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=bf39f342faf613faca7f43b94cc73bc4, ASSIGN 2023-03-05 19:54:39,382 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=40, ppid=39, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=bf39f342faf613faca7f43b94cc73bc4, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,37037,1678046018983; forceNewPlan=false, retain=false 2023-03-05 19:54:39,533 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=40 updating hbase:meta row=bf39f342faf613faca7f43b94cc73bc4, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:39,533 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678046078948.bf39f342faf613faca7f43b94cc73bc4.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1678046079533"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678046079533"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678046079533"}]},"ts":"1678046079533"} 2023-03-05 19:54:39,535 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=41, ppid=40, state=RUNNABLE; OpenRegionProcedure bf39f342faf613faca7f43b94cc73bc4, server=jenkins-hbase4.apache.org,37037,1678046018983}] 2023-03-05 19:54:39,554 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-03-05 19:54:39,690 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678046078948.bf39f342faf613faca7f43b94cc73bc4. 2023-03-05 19:54:39,690 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => bf39f342faf613faca7f43b94cc73bc4, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678046078948.bf39f342faf613faca7f43b94cc73bc4.', STARTKEY => '', ENDKEY => ''} 2023-03-05 19:54:39,690 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly bf39f342faf613faca7f43b94cc73bc4 2023-03-05 19:54:39,691 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678046078948.bf39f342faf613faca7f43b94cc73bc4.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-05 19:54:39,691 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for bf39f342faf613faca7f43b94cc73bc4 2023-03-05 19:54:39,691 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for bf39f342faf613faca7f43b94cc73bc4 2023-03-05 19:54:39,692 INFO [StoreOpener-bf39f342faf613faca7f43b94cc73bc4-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region bf39f342faf613faca7f43b94cc73bc4 2023-03-05 19:54:39,693 DEBUG [StoreOpener-bf39f342faf613faca7f43b94cc73bc4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/family1 2023-03-05 19:54:39,694 DEBUG [StoreOpener-bf39f342faf613faca7f43b94cc73bc4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/family1 2023-03-05 19:54:39,694 INFO [StoreOpener-bf39f342faf613faca7f43b94cc73bc4-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region bf39f342faf613faca7f43b94cc73bc4 columnFamilyName family1 2023-03-05 19:54:39,696 INFO [StoreOpener-bf39f342faf613faca7f43b94cc73bc4-1] regionserver.HStore(310): Store=bf39f342faf613faca7f43b94cc73bc4/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-05 19:54:39,697 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4 2023-03-05 19:54:39,697 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4 2023-03-05 19:54:39,700 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for bf39f342faf613faca7f43b94cc73bc4 2023-03-05 19:54:39,702 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-05 19:54:39,703 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened bf39f342faf613faca7f43b94cc73bc4; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63078154, jitterRate=-0.060062259435653687}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-05 19:54:39,703 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for bf39f342faf613faca7f43b94cc73bc4: 2023-03-05 19:54:39,704 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678046078948.bf39f342faf613faca7f43b94cc73bc4., pid=41, masterSystemTime=1678046079687 2023-03-05 19:54:39,705 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678046078948.bf39f342faf613faca7f43b94cc73bc4. 2023-03-05 19:54:39,705 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678046078948.bf39f342faf613faca7f43b94cc73bc4. 2023-03-05 19:54:39,706 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=40 updating hbase:meta row=bf39f342faf613faca7f43b94cc73bc4, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:39,706 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678046078948.bf39f342faf613faca7f43b94cc73bc4.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1678046079706"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678046079706"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678046079706"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678046079706"}]},"ts":"1678046079706"} 2023-03-05 19:54:39,710 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=41, resume processing ppid=40 2023-03-05 19:54:39,710 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=41, ppid=40, state=SUCCESS; OpenRegionProcedure bf39f342faf613faca7f43b94cc73bc4, server=jenkins-hbase4.apache.org,37037,1678046018983 in 172 msec 2023-03-05 19:54:39,711 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=40, resume processing ppid=39 2023-03-05 19:54:39,712 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=40, ppid=39, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=bf39f342faf613faca7f43b94cc73bc4, ASSIGN in 331 msec 2023-03-05 19:54:39,712 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-05 19:54:39,712 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678046079712"}]},"ts":"1678046079712"} 2023-03-05 19:54:39,714 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, state=ENABLED in hbase:meta 2023-03-05 19:54:39,716 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_POST_OPERATION 2023-03-05 19:54:39,719 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=39, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 769 msec 2023-03-05 19:54:40,055 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-03-05 19:54:40,055 INFO [Listener at localhost/40787] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, procId: 39 completed 2023-03-05 19:54:40,073 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyPolicy/5f33741eb2f558bc4b10a65a4789a550/.tmp/family1/a51784648f1b4ec0b03102fda82fe61f as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyPolicy/5f33741eb2f558bc4b10a65a4789a550/family1/a51784648f1b4ec0b03102fda82fe61f 2023-03-05 19:54:40,078 INFO [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 5f33741eb2f558bc4b10a65a4789a550/family1 of 5f33741eb2f558bc4b10a65a4789a550 into a51784648f1b4ec0b03102fda82fe61f(size=166.4 K), total size for store is 166.4 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-05 19:54:40,078 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 5f33741eb2f558bc4b10a65a4789a550: 2023-03-05 19:54:40,078 INFO [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyPolicy,,1678046065984.5f33741eb2f558bc4b10a65a4789a550., storeName=family1, fileCount=3, fileSize=337.0 K (165.1 K, 6.6 K, 165.4 K), priority=2, time=1678046078834; duration=1sec 2023-03-05 19:54:40,078 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-05 19:54:40,080 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-05 19:54:40,081 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:40,082 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-05 19:54:40,082 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:54:40,082 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-05 19:54:40,082 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-05 19:54:40,082 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:40,082 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:40,083 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:54:40,083 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:40,084 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:54:40,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:54:40,084 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:40,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:40,084 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-05 19:54:40,084 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:40,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-05 19:54:40,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:40,085 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:40,085 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:40,085 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-05 19:54:40,085 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:54:40,085 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-05 19:54:40,085 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-05 19:54:40,085 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-05 19:54:40,085 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678046078948.bf39f342faf613faca7f43b94cc73bc4. 2023-03-05 19:54:40,085 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678046078948.bf39f342faf613faca7f43b94cc73bc4. started... 2023-03-05 19:54:40,086 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing bf39f342faf613faca7f43b94cc73bc4 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-03-05 19:54:40,101 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyPolicy/5f33741eb2f558bc4b10a65a4789a550/family1/eacfc7de27e843fb82f526dd3186b300, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyPolicy/5f33741eb2f558bc4b10a65a4789a550/family1/597f5ec376bb4939a61211a45ea64e95, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyPolicy/5f33741eb2f558bc4b10a65a4789a550/family1/4adcfd5727cf4d62abe246c542ecff67_SeqId_47_] to archive 2023-03-05 19:54:40,102 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-05 19:54:40,103 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyPolicy/5f33741eb2f558bc4b10a65a4789a550/family1/eacfc7de27e843fb82f526dd3186b300 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionWithWeeklyPolicy/5f33741eb2f558bc4b10a65a4789a550/family1/eacfc7de27e843fb82f526dd3186b300 2023-03-05 19:54:40,105 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyPolicy/5f33741eb2f558bc4b10a65a4789a550/family1/597f5ec376bb4939a61211a45ea64e95 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionWithWeeklyPolicy/5f33741eb2f558bc4b10a65a4789a550/family1/597f5ec376bb4939a61211a45ea64e95 2023-03-05 19:54:40,106 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyPolicy/5f33741eb2f558bc4b10a65a4789a550/family1/4adcfd5727cf4d62abe246c542ecff67_SeqId_47_ to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionWithWeeklyPolicy/5f33741eb2f558bc4b10a65a4789a550/family1/4adcfd5727cf4d62abe246c542ecff67_SeqId_47_ 2023-03-05 19:54:40,513 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511309e238e586774429485724a29291fa2b6 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201511309e238e586774429485724a29291fa2b6 2023-03-05 19:54:40,922 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/.tmp/family1/671469e3beed4f10962df0761c25d458 2023-03-05 19:54:40,927 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/.tmp/family1/671469e3beed4f10962df0761c25d458 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/family1/671469e3beed4f10962df0761c25d458 2023-03-05 19:54:40,931 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/family1/671469e3beed4f10962df0761c25d458, entries=1000, sequenceid=5, filesize=191.4 K 2023-03-05 19:54:40,932 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for bf39f342faf613faca7f43b94cc73bc4 in 846ms, sequenceid=5, compaction requested=false 2023-03-05 19:54:40,932 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-05 19:54:40,933 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for bf39f342faf613faca7f43b94cc73bc4: 2023-03-05 19:54:40,933 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678046078948.bf39f342faf613faca7f43b94cc73bc4. 2023-03-05 19:54:40,933 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-05 19:54:40,933 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-05 19:54:40,933 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:40,933 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-05 19:54:40,933 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-05 19:54:40,935 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:40,935 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:40,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:40,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:40,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:40,935 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:40,936 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-05 19:54:40,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:40,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:40,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:40,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:40,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:40,937 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-05 19:54:40,937 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6de42440[Count = 0] remaining members to acquire global barrier 2023-03-05 19:54:40,937 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-05 19:54:40,937 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:40,938 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:40,938 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:40,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:40,938 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-05 19:54:40,938 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-05 19:54:40,938 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,37037,1678046018983' in zk 2023-03-05 19:54:40,938 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:40,938 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-05 19:54:40,940 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:40,940 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-05 19:54:40,940 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:54:40,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:40,940 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-05 19:54:40,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:40,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:40,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:40,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:40,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:40,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:40,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:40,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:40,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:40,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,37037,1678046018983': 2023-03-05 19:54:40,942 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-05 19:54:40,942 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-05 19:54:40,942 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-05 19:54:40,942 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-05 19:54:40,942 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:40,942 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-05 19:54:40,944 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:40,944 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:54:40,944 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:40,944 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:40,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:40,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:40,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:40,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:40,944 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:54:40,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:54:40,944 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:40,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:40,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:40,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:40,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:40,946 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:40,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:40,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:40,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:40,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:40,949 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:40,949 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:54:40,949 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:40,949 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:54:40,949 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-05 19:54:40,949 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-05 19:54:40,949 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:40,949 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:54:40,949 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:54:40,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:54:40,949 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:40,950 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-03-05 19:54:40,950 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:40,950 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:40,950 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-05 19:54:40,950 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-05 19:54:40,950 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:40,950 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:54:40,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:54:41,050 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-05 19:54:41,051 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-05 19:54:41,054 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-05 19:54:41,056 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-03-05 19:54:41,056 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:41,056 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-05 19:54:41,056 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:54:41,056 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-05 19:54:41,056 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-05 19:54:41,057 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:41,057 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:41,058 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:41,058 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:54:41,058 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:54:41,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:54:41,058 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:41,058 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-05 19:54:41,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:41,059 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:41,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-05 19:54:41,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:41,059 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:41,059 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-03-05 19:54:41,060 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:41,060 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-05 19:54:41,060 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:54:41,060 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-05 19:54:41,060 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-05 19:54:41,060 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-05 19:54:41,060 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678046078948.bf39f342faf613faca7f43b94cc73bc4. 2023-03-05 19:54:41,060 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678046078948.bf39f342faf613faca7f43b94cc73bc4. started... 2023-03-05 19:54:41,060 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing bf39f342faf613faca7f43b94cc73bc4 1/1 column families, dataSize=74 B heapSize=408 B 2023-03-05 19:54:41,190 DEBUG [regionserver/jenkins-hbase4:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-03-05 19:54:41,478 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015112832c982860ed24b92943729ef102f47a5 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015112832c982860ed24b92943729ef102f47a5 2023-03-05 19:54:41,884 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/.tmp/family1/7b580c5aefad4c738137e91d405ad19a 2023-03-05 19:54:41,889 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/.tmp/family1/7b580c5aefad4c738137e91d405ad19a as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/family1/7b580c5aefad4c738137e91d405ad19a 2023-03-05 19:54:41,893 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/family1/7b580c5aefad4c738137e91d405ad19a, entries=1, sequenceid=9, filesize=5.0 K 2023-03-05 19:54:41,893 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for bf39f342faf613faca7f43b94cc73bc4 in 833ms, sequenceid=9, compaction requested=false 2023-03-05 19:54:41,893 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for bf39f342faf613faca7f43b94cc73bc4: 2023-03-05 19:54:41,893 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678046078948.bf39f342faf613faca7f43b94cc73bc4. 2023-03-05 19:54:41,894 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-05 19:54:41,894 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-05 19:54:41,894 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:41,894 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-05 19:54:41,894 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-05 19:54:41,895 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:41,895 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:41,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:41,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:41,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:41,896 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:41,896 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-05 19:54:41,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:41,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:41,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:41,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:41,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:41,897 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-05 19:54:41,897 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@51120485[Count = 0] remaining members to acquire global barrier 2023-03-05 19:54:41,897 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-05 19:54:41,897 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:41,898 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:41,898 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:41,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:41,899 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-05 19:54:41,899 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-05 19:54:41,899 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:41,899 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-05 19:54:41,899 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,37037,1678046018983' in zk 2023-03-05 19:54:41,902 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-05 19:54:41,902 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:41,902 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:54:41,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:41,902 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-05 19:54:41,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:41,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:41,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:41,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:41,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:41,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:41,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:41,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:41,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:41,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,37037,1678046018983': 2023-03-05 19:54:41,904 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-05 19:54:41,904 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-05 19:54:41,904 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-05 19:54:41,904 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-05 19:54:41,904 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:41,904 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-05 19:54:41,906 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:41,906 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:41,906 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:41,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:41,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:41,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:41,906 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:54:41,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:41,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:41,907 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:41,907 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:54:41,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:54:41,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:41,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:41,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:41,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:41,908 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:41,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:41,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:41,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:41,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:41,914 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:41,914 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:54:41,914 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:41,914 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:54:41,914 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-03-05 19:54:41,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:54:41,914 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-05 19:54:41,914 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-05 19:54:41,914 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:54:41,914 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:41,914 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:54:41,914 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:41,914 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:41,914 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:41,914 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:54:41,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:54:41,914 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-05 19:54:41,915 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-05 19:54:41,914 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:42,015 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-05 19:54:42,015 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-05 19:54:42,019 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-05 19:54:42,021 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-03-05 19:54:42,021 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:42,021 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-05 19:54:42,021 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:54:42,021 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-05 19:54:42,021 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-05 19:54:42,021 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:42,021 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:42,023 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:54:42,023 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:42,023 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:54:42,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:54:42,023 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:42,023 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-05 19:54:42,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:42,024 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:42,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-05 19:54:42,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:42,024 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:42,024 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-03-05 19:54:42,024 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:42,024 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-05 19:54:42,024 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:54:42,024 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-05 19:54:42,024 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-05 19:54:42,024 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-05 19:54:42,025 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678046078948.bf39f342faf613faca7f43b94cc73bc4. 2023-03-05 19:54:42,025 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678046078948.bf39f342faf613faca7f43b94cc73bc4. started... 2023-03-05 19:54:42,025 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing bf39f342faf613faca7f43b94cc73bc4 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-05 19:54:42,439 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512010e74fc50fe6344c9bf4cc29b51c96a84 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512010e74fc50fe6344c9bf4cc29b51c96a84 2023-03-05 19:54:42,845 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/.tmp/family1/10f3e25324cd4f92875dd59bd77aa3d9 2023-03-05 19:54:42,850 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/.tmp/family1/10f3e25324cd4f92875dd59bd77aa3d9 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/family1/10f3e25324cd4f92875dd59bd77aa3d9 2023-03-05 19:54:42,854 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/family1/10f3e25324cd4f92875dd59bd77aa3d9, entries=1, sequenceid=13, filesize=5.0 K 2023-03-05 19:54:42,854 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for bf39f342faf613faca7f43b94cc73bc4 in 829ms, sequenceid=13, compaction requested=true 2023-03-05 19:54:42,855 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for bf39f342faf613faca7f43b94cc73bc4: 2023-03-05 19:54:42,855 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678046078948.bf39f342faf613faca7f43b94cc73bc4. 2023-03-05 19:54:42,855 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-05 19:54:42,855 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-05 19:54:42,855 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:42,855 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-05 19:54:42,855 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-05 19:54:42,857 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:42,857 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:42,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:42,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:42,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:42,858 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:42,858 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-05 19:54:42,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:42,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:42,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:42,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:42,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:42,859 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-05 19:54:42,859 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3cc6463a[Count = 0] remaining members to acquire global barrier 2023-03-05 19:54:42,859 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-05 19:54:42,859 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:42,860 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:42,860 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:42,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:42,861 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-05 19:54:42,861 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:42,861 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-05 19:54:42,861 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-05 19:54:42,861 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,37037,1678046018983' in zk 2023-03-05 19:54:42,862 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-05 19:54:42,862 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:42,862 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:54:42,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:42,863 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-05 19:54:42,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:42,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:42,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:42,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:42,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:42,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:42,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:42,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:42,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:42,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,37037,1678046018983': 2023-03-05 19:54:42,864 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-05 19:54:42,864 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-05 19:54:42,865 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-05 19:54:42,865 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-05 19:54:42,865 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:42,865 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-05 19:54:42,866 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:42,866 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:42,866 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:42,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:42,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:42,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:42,866 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:54:42,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:42,867 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:42,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:42,867 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:54:42,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:54:42,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:42,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:42,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:42,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:42,868 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:42,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:42,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:42,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:42,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:42,871 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:54:42,871 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:42,871 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:54:42,871 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-05 19:54:42,871 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-05 19:54:42,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:54:42,871 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:54:42,871 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-03-05 19:54:42,871 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:42,871 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:54:42,872 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-05 19:54:42,872 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:42,872 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:54:42,872 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:42,872 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-05 19:54:42,872 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:42,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:54:42,872 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:42,872 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:42,972 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-05 19:54:42,973 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-05 19:54:42,976 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-05 19:54:42,978 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-03-05 19:54:42,978 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:42,978 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-05 19:54:42,978 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:54:42,978 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-05 19:54:42,979 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-05 19:54:42,979 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:42,979 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:42,980 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:42,980 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:54:42,980 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:54:42,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:54:42,980 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:42,980 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-05 19:54:42,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:42,981 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:42,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-05 19:54:42,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:42,981 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:42,981 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-03-05 19:54:42,981 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:42,982 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-05 19:54:42,982 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:54:42,982 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-05 19:54:42,982 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-05 19:54:42,982 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-05 19:54:42,982 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678046078948.bf39f342faf613faca7f43b94cc73bc4. 2023-03-05 19:54:42,982 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678046078948.bf39f342faf613faca7f43b94cc73bc4. started... 2023-03-05 19:54:42,982 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing bf39f342faf613faca7f43b94cc73bc4 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-05 19:54:43,395 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205d8269d05a54c4de69f2129d205cab5f6 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151205d8269d05a54c4de69f2129d205cab5f6 2023-03-05 19:54:43,801 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/.tmp/family1/ff53d888242a465994b3e3af12d17bbc 2023-03-05 19:54:43,806 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/.tmp/family1/ff53d888242a465994b3e3af12d17bbc as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/family1/ff53d888242a465994b3e3af12d17bbc 2023-03-05 19:54:43,809 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/family1/ff53d888242a465994b3e3af12d17bbc, entries=1, sequenceid=17, filesize=5.0 K 2023-03-05 19:54:43,810 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for bf39f342faf613faca7f43b94cc73bc4 in 828ms, sequenceid=17, compaction requested=true 2023-03-05 19:54:43,810 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for bf39f342faf613faca7f43b94cc73bc4: 2023-03-05 19:54:43,810 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678046078948.bf39f342faf613faca7f43b94cc73bc4. 2023-03-05 19:54:43,810 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-05 19:54:43,810 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-05 19:54:43,810 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:43,810 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-05 19:54:43,810 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-05 19:54:43,812 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:43,812 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:43,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:43,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:43,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:43,812 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:43,812 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-05 19:54:43,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:43,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:43,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:43,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:43,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:43,814 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-05 19:54:43,814 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1ced665f[Count = 0] remaining members to acquire global barrier 2023-03-05 19:54:43,814 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-05 19:54:43,814 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:43,815 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:43,815 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:43,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:43,815 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-05 19:54:43,815 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-05 19:54:43,815 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,37037,1678046018983' in zk 2023-03-05 19:54:43,815 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:43,815 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-05 19:54:43,818 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-05 19:54:43,818 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:43,818 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:54:43,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:43,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:43,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:43,818 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-05 19:54:43,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:43,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:43,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:43,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:43,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:43,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:43,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:43,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,37037,1678046018983': 2023-03-05 19:54:43,820 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-05 19:54:43,820 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-05 19:54:43,820 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-05 19:54:43,820 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-05 19:54:43,820 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:43,820 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-05 19:54:43,821 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:43,821 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:43,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:43,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:43,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:43,821 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:43,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:43,821 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:54:43,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:43,822 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:54:43,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:54:43,822 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:43,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:43,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:43,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:43,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:43,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:43,823 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:43,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:43,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:43,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:43,825 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:43,825 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:54:43,825 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:43,825 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:54:43,825 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-05 19:54:43,825 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-05 19:54:43,825 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:43,825 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:54:43,825 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:54:43,825 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:43,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:54:43,825 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-05 19:54:43,825 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:43,825 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-05 19:54:43,825 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:54:43,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:54:43,825 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:43,825 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:43,925 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-05 19:54:43,926 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-05 19:54:43,930 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-05 19:54:43,932 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-03-05 19:54:43,932 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:43,932 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-05 19:54:43,932 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:54:43,932 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-05 19:54:43,932 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-05 19:54:43,932 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:43,932 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:43,934 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:43,934 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:54:43,934 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:54:43,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:54:43,934 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:43,934 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-05 19:54:43,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:43,935 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:43,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-05 19:54:43,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:43,935 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:43,935 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-03-05 19:54:43,935 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:43,935 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-05 19:54:43,935 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:54:43,935 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-05 19:54:43,936 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-05 19:54:43,936 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-05 19:54:43,936 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678046078948.bf39f342faf613faca7f43b94cc73bc4. 2023-03-05 19:54:43,936 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678046078948.bf39f342faf613faca7f43b94cc73bc4. started... 2023-03-05 19:54:43,936 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing bf39f342faf613faca7f43b94cc73bc4 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-05 19:54:44,348 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512288eff7c2d1a364260967d2be88f2da887 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512288eff7c2d1a364260967d2be88f2da887 2023-03-05 19:54:44,754 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/.tmp/family1/14ff3ec7787640e0930bc1261c87a834 2023-03-05 19:54:44,758 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/.tmp/family1/14ff3ec7787640e0930bc1261c87a834 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/family1/14ff3ec7787640e0930bc1261c87a834 2023-03-05 19:54:44,762 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/family1/14ff3ec7787640e0930bc1261c87a834, entries=1, sequenceid=21, filesize=5.0 K 2023-03-05 19:54:44,762 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for bf39f342faf613faca7f43b94cc73bc4 in 826ms, sequenceid=21, compaction requested=true 2023-03-05 19:54:44,762 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for bf39f342faf613faca7f43b94cc73bc4: 2023-03-05 19:54:44,763 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678046078948.bf39f342faf613faca7f43b94cc73bc4. 2023-03-05 19:54:44,763 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-05 19:54:44,763 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-05 19:54:44,763 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:44,763 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-05 19:54:44,763 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-05 19:54:44,767 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:44,767 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:44,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:44,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:44,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:44,767 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:44,767 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-05 19:54:44,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:44,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:44,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:44,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:44,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:44,769 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-05 19:54:44,769 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-05 19:54:44,769 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@50e787a6[Count = 0] remaining members to acquire global barrier 2023-03-05 19:54:44,769 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:44,770 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:44,770 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:44,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:44,770 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-05 19:54:44,770 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-05 19:54:44,770 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,37037,1678046018983' in zk 2023-03-05 19:54:44,770 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:44,770 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-05 19:54:44,771 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:44,771 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-05 19:54:44,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:44,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:44,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:44,771 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:54:44,771 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-05 19:54:44,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:44,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:44,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:44,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:44,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:44,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:44,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:44,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,37037,1678046018983': 2023-03-05 19:54:44,773 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-05 19:54:44,773 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-05 19:54:44,773 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-05 19:54:44,773 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-05 19:54:44,773 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:44,773 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-05 19:54:44,776 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:44,776 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:44,776 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:54:44,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:44,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:44,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:44,776 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:44,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:44,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:44,776 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:54:44,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:54:44,776 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:44,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:44,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:44,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:44,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:44,777 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:44,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:44,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:44,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:44,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:44,779 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:44,779 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:44,779 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:44,779 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:54:44,780 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:44,780 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-05 19:54:44,780 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-03-05 19:54:44,780 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-05 19:54:44,780 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:44,780 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:54:44,780 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:44,780 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:54:44,780 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:54:44,780 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:44,780 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-05 19:54:44,780 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-05 19:54:44,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:54:44,780 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:54:44,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:54:44,880 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-05 19:54:44,881 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-05 19:54:44,885 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-05 19:54:44,886 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-03-05 19:54:44,886 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:44,886 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-05 19:54:44,886 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:54:44,886 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-05 19:54:44,887 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-05 19:54:44,887 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:44,887 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:44,888 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:44,888 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:54:44,888 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:54:44,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:54:44,888 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:44,888 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-05 19:54:44,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:44,889 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:44,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-05 19:54:44,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:44,889 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:44,889 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-03-05 19:54:44,889 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:44,889 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-05 19:54:44,889 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:54:44,889 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-05 19:54:44,890 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-05 19:54:44,890 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-05 19:54:44,890 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678046078948.bf39f342faf613faca7f43b94cc73bc4. 2023-03-05 19:54:44,890 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678046078948.bf39f342faf613faca7f43b94cc73bc4. started... 2023-03-05 19:54:44,890 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing bf39f342faf613faca7f43b94cc73bc4 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-05 19:54:45,217 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-05 19:54:45,301 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512310808d5d2d7a84aa2aa2411592002d596 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512310808d5d2d7a84aa2aa2411592002d596 2023-03-05 19:54:45,707 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/.tmp/family1/ac2629f2e99c4ee494d00a000a43748b 2023-03-05 19:54:45,711 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/.tmp/family1/ac2629f2e99c4ee494d00a000a43748b as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/family1/ac2629f2e99c4ee494d00a000a43748b 2023-03-05 19:54:45,715 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/family1/ac2629f2e99c4ee494d00a000a43748b, entries=1, sequenceid=25, filesize=5.0 K 2023-03-05 19:54:45,716 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for bf39f342faf613faca7f43b94cc73bc4 in 826ms, sequenceid=25, compaction requested=true 2023-03-05 19:54:45,716 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for bf39f342faf613faca7f43b94cc73bc4: 2023-03-05 19:54:45,716 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678046078948.bf39f342faf613faca7f43b94cc73bc4. 2023-03-05 19:54:45,716 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-05 19:54:45,716 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-05 19:54:45,716 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:45,716 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-05 19:54:45,716 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-05 19:54:45,718 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:45,718 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:45,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:45,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:45,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:45,718 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:45,718 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-05 19:54:45,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:45,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:45,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:45,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:45,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:45,719 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-05 19:54:45,719 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@11908713[Count = 0] remaining members to acquire global barrier 2023-03-05 19:54:45,719 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-05 19:54:45,719 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:45,721 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:45,721 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:45,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:45,722 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-05 19:54:45,722 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-05 19:54:45,722 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,37037,1678046018983' in zk 2023-03-05 19:54:45,722 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:45,722 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-05 19:54:45,723 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:45,723 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-05 19:54:45,723 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:54:45,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:45,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:45,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:45,723 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-05 19:54:45,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:45,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:45,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:45,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:45,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:45,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:45,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:45,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,37037,1678046018983': 2023-03-05 19:54:45,725 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-05 19:54:45,725 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-05 19:54:45,725 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-05 19:54:45,725 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-05 19:54:45,725 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:45,725 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-05 19:54:45,726 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:45,726 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:45,727 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:54:45,727 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:45,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:45,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:45,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:45,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:45,727 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:54:45,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:54:45,727 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:45,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:45,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:45,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:45,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:45,728 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:45,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:45,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:45,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:45,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:45,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:45,730 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:45,730 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:45,730 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:54:45,730 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:45,730 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:54:45,730 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:54:45,730 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:45,730 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:45,730 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:45,730 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:54:45,730 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:45,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:54:45,730 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-05 19:54:45,730 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-05 19:54:45,730 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-03-05 19:54:45,731 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:54:45,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:54:45,731 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-05 19:54:45,731 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-05 19:54:45,831 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-05 19:54:45,832 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-05 19:54:45,835 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-05 19:54:45,837 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-03-05 19:54:45,837 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:45,837 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-05 19:54:45,837 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:54:45,837 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-05 19:54:45,837 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-05 19:54:45,837 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:45,837 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:45,840 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:45,840 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:54:45,840 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:54:45,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:54:45,840 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:45,840 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-05 19:54:45,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:45,840 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:45,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-05 19:54:45,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:45,841 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:45,841 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-03-05 19:54:45,841 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:45,841 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-05 19:54:45,841 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:54:45,841 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-05 19:54:45,841 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-05 19:54:45,841 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-05 19:54:45,841 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678046078948.bf39f342faf613faca7f43b94cc73bc4. 2023-03-05 19:54:45,841 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678046078948.bf39f342faf613faca7f43b94cc73bc4. started... 2023-03-05 19:54:45,841 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing bf39f342faf613faca7f43b94cc73bc4 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-05 19:54:46,259 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601010bb9a56654414ebf8f1cef01dc0586b8 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201601010bb9a56654414ebf8f1cef01dc0586b8 2023-03-05 19:54:46,664 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/.tmp/family1/cb5ab59bb2ae41e5bb229de643842ab3 2023-03-05 19:54:46,670 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/.tmp/family1/cb5ab59bb2ae41e5bb229de643842ab3 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/family1/cb5ab59bb2ae41e5bb229de643842ab3 2023-03-05 19:54:46,675 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/family1/cb5ab59bb2ae41e5bb229de643842ab3, entries=1, sequenceid=29, filesize=5.0 K 2023-03-05 19:54:46,676 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for bf39f342faf613faca7f43b94cc73bc4 in 835ms, sequenceid=29, compaction requested=true 2023-03-05 19:54:46,677 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for bf39f342faf613faca7f43b94cc73bc4: 2023-03-05 19:54:46,677 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678046078948.bf39f342faf613faca7f43b94cc73bc4. 2023-03-05 19:54:46,677 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-05 19:54:46,677 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-05 19:54:46,677 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:46,677 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-05 19:54:46,677 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-05 19:54:46,678 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:46,678 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:46,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:46,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:46,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:46,679 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:46,679 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-05 19:54:46,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:46,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:46,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:46,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:46,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:46,680 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-05 19:54:46,680 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1fe9f48d[Count = 0] remaining members to acquire global barrier 2023-03-05 19:54:46,680 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-05 19:54:46,680 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:46,681 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:46,681 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:46,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:46,681 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:46,681 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-05 19:54:46,681 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-05 19:54:46,681 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-05 19:54:46,681 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,37037,1678046018983' in zk 2023-03-05 19:54:46,683 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:46,683 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-05 19:54:46,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:46,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:46,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:46,683 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:54:46,683 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-05 19:54:46,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:46,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:46,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:46,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:46,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:46,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:46,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:46,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,37037,1678046018983': 2023-03-05 19:54:46,685 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-05 19:54:46,685 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-05 19:54:46,685 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-05 19:54:46,685 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-05 19:54:46,685 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:46,685 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-05 19:54:46,689 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:46,689 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:46,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:46,690 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:46,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:46,690 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:54:46,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:46,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:46,690 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:46,690 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:54:46,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:54:46,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:46,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:46,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:46,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:46,691 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:46,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:46,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:46,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:46,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:46,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:46,693 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:46,693 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:54:46,693 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:46,693 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:54:46,693 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:54:46,693 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:46,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:54:46,693 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:46,693 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-05 19:54:46,693 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-05 19:54:46,693 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:54:46,693 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-03-05 19:54:46,693 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:54:46,693 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:46,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:54:46,694 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-05 19:54:46,694 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-05 19:54:46,694 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:46,694 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:46,794 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-05 19:54:46,795 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-05 19:54:46,798 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-05 19:54:46,800 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-03-05 19:54:46,800 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:46,800 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-05 19:54:46,800 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:54:46,800 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-05 19:54:46,800 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-05 19:54:46,800 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:46,800 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:46,802 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:46,802 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:54:46,802 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:54:46,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:54:46,802 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:46,802 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-05 19:54:46,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:46,802 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:46,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-05 19:54:46,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:46,803 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:46,803 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-03-05 19:54:46,803 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:46,803 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-05 19:54:46,803 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:54:46,803 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-05 19:54:46,803 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-05 19:54:46,803 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-05 19:54:46,803 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678046078948.bf39f342faf613faca7f43b94cc73bc4. 2023-03-05 19:54:46,803 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678046078948.bf39f342faf613faca7f43b94cc73bc4. started... 2023-03-05 19:54:46,803 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing bf39f342faf613faca7f43b94cc73bc4 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-05 19:54:47,215 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103456eae0bed22480eb3242f479bb800ac to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20160103456eae0bed22480eb3242f479bb800ac 2023-03-05 19:54:47,620 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/.tmp/family1/058b831671074f5ca1fbc5fc1063227f 2023-03-05 19:54:47,625 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/.tmp/family1/058b831671074f5ca1fbc5fc1063227f as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/family1/058b831671074f5ca1fbc5fc1063227f 2023-03-05 19:54:47,630 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/family1/058b831671074f5ca1fbc5fc1063227f, entries=1, sequenceid=33, filesize=5.0 K 2023-03-05 19:54:47,631 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for bf39f342faf613faca7f43b94cc73bc4 in 828ms, sequenceid=33, compaction requested=true 2023-03-05 19:54:47,631 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for bf39f342faf613faca7f43b94cc73bc4: 2023-03-05 19:54:47,631 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678046078948.bf39f342faf613faca7f43b94cc73bc4. 2023-03-05 19:54:47,631 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-05 19:54:47,631 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-05 19:54:47,631 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:47,631 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-05 19:54:47,631 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-05 19:54:47,633 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:47,633 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:47,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:47,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:47,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:47,634 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:47,634 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-05 19:54:47,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:47,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:47,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:47,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:47,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:47,636 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-05 19:54:47,636 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6841b6b1[Count = 0] remaining members to acquire global barrier 2023-03-05 19:54:47,636 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-05 19:54:47,636 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:47,638 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:47,638 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:47,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:47,639 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-05 19:54:47,639 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:47,639 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-05 19:54:47,639 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-05 19:54:47,639 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,37037,1678046018983' in zk 2023-03-05 19:54:47,640 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:47,640 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-05 19:54:47,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:47,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:47,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:47,640 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:54:47,640 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-05 19:54:47,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:47,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:47,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:47,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:47,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:47,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:47,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:47,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,37037,1678046018983': 2023-03-05 19:54:47,642 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-05 19:54:47,642 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-05 19:54:47,642 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-05 19:54:47,643 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-05 19:54:47,643 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:47,643 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-05 19:54:47,644 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:47,644 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:47,644 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:54:47,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:47,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:47,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:47,644 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:47,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:47,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:47,645 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:47,645 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:54:47,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:54:47,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:47,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:47,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:47,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:47,647 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:47,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:47,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:47,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:47,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:47,649 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:47,649 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:54:47,649 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:47,649 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:54:47,649 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-05 19:54:47,649 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-05 19:54:47,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:54:47,649 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:54:47,649 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:47,650 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:47,650 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:47,650 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:47,649 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:54:47,650 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-05 19:54:47,650 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-05 19:54:47,650 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:47,650 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:54:47,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:54:47,652 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-03-05 19:54:47,750 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-05 19:54:47,751 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-05 19:54:47,755 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-05 19:54:47,756 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-03-05 19:54:47,756 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:47,756 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-05 19:54:47,756 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:54:47,756 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-05 19:54:47,756 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-05 19:54:47,757 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:47,757 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:47,759 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:54:47,759 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:47,759 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:54:47,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:54:47,759 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:47,759 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-05 19:54:47,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:47,760 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:47,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-05 19:54:47,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:47,760 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:47,760 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-03-05 19:54:47,760 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:47,760 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-05 19:54:47,760 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:54:47,760 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-05 19:54:47,761 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-05 19:54:47,761 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-05 19:54:47,761 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678046078948.bf39f342faf613faca7f43b94cc73bc4. 2023-03-05 19:54:47,761 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678046078948.bf39f342faf613faca7f43b94cc73bc4. started... 2023-03-05 19:54:47,761 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing bf39f342faf613faca7f43b94cc73bc4 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-05 19:54:48,173 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20150907b0f351d987204a39a3c3f22ace20355a to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20150907b0f351d987204a39a3c3f22ace20355a 2023-03-05 19:54:48,579 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/.tmp/family1/b1c9923551304bada6fe356e5acfd6b0 2023-03-05 19:54:48,584 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/.tmp/family1/b1c9923551304bada6fe356e5acfd6b0 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/family1/b1c9923551304bada6fe356e5acfd6b0 2023-03-05 19:54:48,589 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/family1/b1c9923551304bada6fe356e5acfd6b0, entries=1, sequenceid=37, filesize=5.0 K 2023-03-05 19:54:48,590 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for bf39f342faf613faca7f43b94cc73bc4 in 829ms, sequenceid=37, compaction requested=true 2023-03-05 19:54:48,590 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for bf39f342faf613faca7f43b94cc73bc4: 2023-03-05 19:54:48,590 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678046078948.bf39f342faf613faca7f43b94cc73bc4. 2023-03-05 19:54:48,590 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-05 19:54:48,590 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-05 19:54:48,590 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:48,590 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-05 19:54:48,590 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-05 19:54:48,592 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:48,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:48,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:48,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:48,592 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:48,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:48,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:48,593 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:48,593 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-05 19:54:48,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:48,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:48,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:48,595 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-05 19:54:48,595 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@477d6a8[Count = 0] remaining members to acquire global barrier 2023-03-05 19:54:48,595 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-05 19:54:48,595 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:48,596 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:48,596 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:48,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:48,596 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-05 19:54:48,596 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-05 19:54:48,596 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,37037,1678046018983' in zk 2023-03-05 19:54:48,597 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:48,597 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-05 19:54:48,598 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-05 19:54:48,598 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:54:48,598 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-05 19:54:48,598 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:48,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:48,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:48,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:48,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:48,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:48,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:48,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:48,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:48,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:48,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:48,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,37037,1678046018983': 2023-03-05 19:54:48,601 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-05 19:54:48,601 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-05 19:54:48,601 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-05 19:54:48,601 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-05 19:54:48,601 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:48,601 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-05 19:54:48,605 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:48,605 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:54:48,605 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:48,605 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:48,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:48,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:48,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:48,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:48,606 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:54:48,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:54:48,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:48,606 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:48,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:48,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:48,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:48,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:48,607 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:48,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:48,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:48,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:48,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:48,613 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:48,613 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:48,613 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:54:48,613 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:48,613 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:48,613 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:48,613 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:48,613 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:54:48,614 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:54:48,614 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-05 19:54:48,614 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-05 19:54:48,613 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:54:48,614 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-03-05 19:54:48,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:54:48,614 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:48,614 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-05 19:54:48,614 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-05 19:54:48,615 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:54:48,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:54:48,714 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-05 19:54:48,715 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-05 19:54:48,719 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-05 19:54:48,721 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-03-05 19:54:48,721 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:48,721 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-05 19:54:48,721 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:54:48,721 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-05 19:54:48,721 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-05 19:54:48,722 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:48,722 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:48,723 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:54:48,723 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:48,723 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:54:48,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:54:48,723 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:48,724 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-05 19:54:48,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:48,724 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:48,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-05 19:54:48,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:48,724 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:48,725 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-03-05 19:54:48,725 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:48,725 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-05 19:54:48,725 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:54:48,725 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-05 19:54:48,725 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-05 19:54:48,725 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-05 19:54:48,725 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678046078948.bf39f342faf613faca7f43b94cc73bc4. 2023-03-05 19:54:48,725 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678046078948.bf39f342faf613faca7f43b94cc73bc4. started... 2023-03-05 19:54:48,725 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing bf39f342faf613faca7f43b94cc73bc4 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-05 19:54:49,140 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015112019abd20e29104d17badd7082788f293b to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015112019abd20e29104d17badd7082788f293b 2023-03-05 19:54:49,554 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/.tmp/family1/50f9ccb3289147dc9937be82226b39b2 2023-03-05 19:54:49,559 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/.tmp/family1/50f9ccb3289147dc9937be82226b39b2 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/family1/50f9ccb3289147dc9937be82226b39b2 2023-03-05 19:54:49,564 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/family1/50f9ccb3289147dc9937be82226b39b2, entries=1, sequenceid=41, filesize=5.0 K 2023-03-05 19:54:49,565 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for bf39f342faf613faca7f43b94cc73bc4 in 840ms, sequenceid=41, compaction requested=true 2023-03-05 19:54:49,565 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for bf39f342faf613faca7f43b94cc73bc4: 2023-03-05 19:54:49,565 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678046078948.bf39f342faf613faca7f43b94cc73bc4. 2023-03-05 19:54:49,565 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-05 19:54:49,565 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-05 19:54:49,565 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:49,565 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-05 19:54:49,565 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-05 19:54:49,567 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:49,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:49,567 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:49,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:49,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:49,568 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:49,568 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-05 19:54:49,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:49,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:49,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:49,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:49,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:49,569 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-05 19:54:49,570 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@11ca712a[Count = 0] remaining members to acquire global barrier 2023-03-05 19:54:49,570 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-05 19:54:49,570 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:49,571 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:49,571 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:49,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:49,571 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-05 19:54:49,571 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-05 19:54:49,571 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,37037,1678046018983' in zk 2023-03-05 19:54:49,571 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:49,571 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-05 19:54:49,572 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:49,572 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-05 19:54:49,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:49,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:49,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:49,572 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:54:49,572 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-05 19:54:49,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:49,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:49,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:49,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:49,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:49,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:49,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:49,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,37037,1678046018983': 2023-03-05 19:54:49,574 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-05 19:54:49,574 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-05 19:54:49,574 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-05 19:54:49,574 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-05 19:54:49,574 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:49,574 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-05 19:54:49,576 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:49,576 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:49,576 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:49,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:49,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:49,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:49,576 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:54:49,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:49,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:49,576 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:49,576 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:54:49,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:54:49,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:49,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:49,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:49,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:49,578 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:49,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:49,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:49,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:49,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:49,580 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:54:49,581 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:54:49,581 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:54:49,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:54:49,581 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:49,581 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:49,581 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:49,581 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:49,581 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:49,581 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:54:49,581 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:54:49,581 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:49,581 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:49,581 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-05 19:54:49,581 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-05 19:54:49,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:54:49,581 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-05 19:54:49,582 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-05 19:54:49,682 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-05 19:54:49,683 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-05 19:54:49,684 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=38221] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:49,685 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] procedure2.ProcedureExecutor(1029): Stored pid=42, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-03-05 19:54:49,686 DEBUG [PEWorker-5] locking.LockProcedure(309): LOCKED pid=42, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-03-05 19:54:49,687 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-05 19:54:49,687 INFO [PEWorker-5] procedure2.TimeoutExecutorThread(81): ADDED pid=42, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678046689687 2023-03-05 19:54:49,690 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-05 19:54:49,691 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column 'family1' 2023-03-05 19:54:49,691 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-05 19:54:49,691 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] zookeeper.ReadOnlyZKClient(139): Connect 0x58345cef to 127.0.0.1:62957 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-05 19:54:49,696 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4caded48, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-05 19:54:49,696 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-05 19:54:50,533 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601034681ae87035a4ade943c735d04cafd52 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201601034681ae87035a4ade943c735d04cafd52 2023-03-05 19:54:50,534 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-05 19:54:50,551 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-05 19:54:50,552 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:48010, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-05 19:54:50,559 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-05 19:54:50,560 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:51082, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-05 19:54:50,569 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/79ea429111da4ee1898175d7b4e91970 first=Optional[r3] last=Optional[r6] 2023-03-05 19:54:50,569 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678046078948.bf39f342faf613faca7f43b94cc73bc4., hostname=jenkins-hbase4.apache.org,37037,1678046018983, seqNum=2 for row with hfile group [{family1,hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/79ea429111da4ee1898175d7b4e91970}] 2023-03-05 19:54:50,574 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(763): Validating hfile at hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/79ea429111da4ee1898175d7b4e91970 for inclusion in bf39f342faf613faca7f43b94cc73bc4/family1 2023-03-05 19:54:50,577 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-03-05 19:54:50,577 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(777): Region bounds: first= last= 2023-03-05 19:54:50,577 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HRegion(2446): Flush status journal for bf39f342faf613faca7f43b94cc73bc4: 2023-03-05 19:54:50,578 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/79ea429111da4ee1898175d7b4e91970 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__fnfstgsuul0rldvfn0gcabfofn53p5378hegs69bcef678spg07pqf5s97ftvnjh/family1/79ea429111da4ee1898175d7b4e91970 2023-03-05 19:54:50,580 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__fnfstgsuul0rldvfn0gcabfofn53p5378hegs69bcef678spg07pqf5s97ftvnjh/family1/79ea429111da4ee1898175d7b4e91970 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/family1/c76304895c3647cb8c7d3b603648ca10_SeqId_44_ 2023-03-05 19:54:50,580 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(849): Loaded HFile hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__fnfstgsuul0rldvfn0gcabfofn53p5378hegs69bcef678spg07pqf5s97ftvnjh/family1/79ea429111da4ee1898175d7b4e91970 into bf39f342faf613faca7f43b94cc73bc4/family1 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/family1/c76304895c3647cb8c7d3b603648ca10_SeqId_44_ - updating store file list. 2023-03-05 19:54:50,583 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(882): Loaded HFile hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/family1/c76304895c3647cb8c7d3b603648ca10_SeqId_44_ into bf39f342faf613faca7f43b94cc73bc4/family1 2023-03-05 19:54:50,584 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(855): Successfully loaded hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__fnfstgsuul0rldvfn0gcabfofn53p5378hegs69bcef678spg07pqf5s97ftvnjh/family1/79ea429111da4ee1898175d7b4e91970 into bf39f342faf613faca7f43b94cc73bc4/family1 (new location: hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/family1/c76304895c3647cb8c7d3b603648ca10_SeqId_44_) 2023-03-05 19:54:50,584 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__fnfstgsuul0rldvfn0gcabfofn53p5378hegs69bcef678spg07pqf5s97ftvnjh/family1/79ea429111da4ee1898175d7b4e91970 2023-03-05 19:54:50,584 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-03-05 19:54:50,587 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51565 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-03-05 19:54:50,587 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(1912): bf39f342faf613faca7f43b94cc73bc4/family1 is initiating minor compaction 2023-03-05 19:54:50,587 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@39bdc8c2; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-05 19:54:50,587 INFO [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of bf39f342faf613faca7f43b94cc73bc4/family1 in testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678046078948.bf39f342faf613faca7f43b94cc73bc4. 2023-03-05 19:54:50,588 INFO [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/family1/7b580c5aefad4c738137e91d405ad19a, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/family1/10f3e25324cd4f92875dd59bd77aa3d9, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/family1/ff53d888242a465994b3e3af12d17bbc, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/family1/14ff3ec7787640e0930bc1261c87a834, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/family1/ac2629f2e99c4ee494d00a000a43748b, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/family1/cb5ab59bb2ae41e5bb229de643842ab3, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/family1/058b831671074f5ca1fbc5fc1063227f, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/family1/b1c9923551304bada6fe356e5acfd6b0, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/family1/50f9ccb3289147dc9937be82226b39b2, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/family1/c76304895c3647cb8c7d3b603648ca10_SeqId_44_] into tmpdir=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/.tmp, totalSize=50.4 K 2023-03-05 19:54:50,588 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.Compactor(207): Compacting 7b580c5aefad4c738137e91d405ad19a, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-03-05 19:54:50,588 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.Compactor(207): Compacting 10f3e25324cd4f92875dd59bd77aa3d9, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=13 2023-03-05 19:54:50,588 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.Compactor(207): Compacting ff53d888242a465994b3e3af12d17bbc, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=17 2023-03-05 19:54:50,588 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.Compactor(207): Compacting 14ff3ec7787640e0930bc1261c87a834, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=21 2023-03-05 19:54:50,588 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.Compactor(207): Compacting ac2629f2e99c4ee494d00a000a43748b, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=25 2023-03-05 19:54:50,588 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.Compactor(207): Compacting cb5ab59bb2ae41e5bb229de643842ab3, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=29 2023-03-05 19:54:50,588 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.Compactor(207): Compacting 058b831671074f5ca1fbc5fc1063227f, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33 2023-03-05 19:54:50,588 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.Compactor(207): Compacting b1c9923551304bada6fe356e5acfd6b0, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=37 2023-03-05 19:54:50,588 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.Compactor(207): Compacting 50f9ccb3289147dc9937be82226b39b2, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=41 2023-03-05 19:54:50,588 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.Compactor(207): Compacting c76304895c3647cb8c7d3b603648ca10_SeqId_44_, keycount=4, bloomtype=ROW, size=5.6 K, encoding=NONE, compression=NONE, seqNum=45 2023-03-05 19:54:50,589 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-05 19:54:50,590 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-05 19:54:50,591 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512288eff7c2d1a364260967d2be88f2da887 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512288eff7c2d1a364260967d2be88f2da887 2023-03-05 19:54:50,593 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512310808d5d2d7a84aa2aa2411592002d596 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512310808d5d2d7a84aa2aa2411592002d596 2023-03-05 19:54:50,594 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201601010bb9a56654414ebf8f1cef01dc0586b8 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201601010bb9a56654414ebf8f1cef01dc0586b8 2023-03-05 19:54:50,595 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20160103456eae0bed22480eb3242f479bb800ac to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20160103456eae0bed22480eb3242f479bb800ac 2023-03-05 19:54:50,595 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 4 expired mob files are deleted 2023-03-05 19:54:50,595 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-03-05 19:54:50,596 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-05 19:54:50,596 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-05 19:54:50,596 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-05 19:54:50,596 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-05 19:54:50,596 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-05 19:54:50,596 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-05 19:54:50,596 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-05 19:54:50,596 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-05 19:54:50,614 INFO [RS:0;jenkins-hbase4:37037-shortCompactions-0] throttle.PressureAwareThroughputController(145): bf39f342faf613faca7f43b94cc73bc4#family1#compaction#60 average throughput is 0.53 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-05 19:54:51,047 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512053cebb9d3827b4e7b9ca0e33e32b5eac7 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512053cebb9d3827b4e7b9ca0e33e32b5eac7 2023-03-05 19:54:51,048 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-05 19:54:51,071 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/c8fafc7831e94aabaece01dbc3984833 first=Optional[r00] last=Optional[r2] 2023-03-05 19:54:51,071 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678046078948.bf39f342faf613faca7f43b94cc73bc4., hostname=jenkins-hbase4.apache.org,37037,1678046018983, seqNum=2 for row with hfile group [{family1,hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/c8fafc7831e94aabaece01dbc3984833}] 2023-03-05 19:54:51,073 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(763): Validating hfile at hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/c8fafc7831e94aabaece01dbc3984833 for inclusion in bf39f342faf613faca7f43b94cc73bc4/family1 2023-03-05 19:54:51,075 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(775): HFile bounds: first=r00 last=r2 2023-03-05 19:54:51,075 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(777): Region bounds: first= last= 2023-03-05 19:54:51,075 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HRegion(2446): Flush status journal for bf39f342faf613faca7f43b94cc73bc4: 2023-03-05 19:54:51,076 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/c8fafc7831e94aabaece01dbc3984833 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__ig76rk901v3kinr9f19u92ehel5d04and1kkgrb5ki1m9oenq0a02jbu7ubvdjg8/family1/c8fafc7831e94aabaece01dbc3984833 2023-03-05 19:54:51,078 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__ig76rk901v3kinr9f19u92ehel5d04and1kkgrb5ki1m9oenq0a02jbu7ubvdjg8/family1/c8fafc7831e94aabaece01dbc3984833 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/family1/a840e944a61f4f37b402c9025eb3b181_SeqId_46_ 2023-03-05 19:54:51,079 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(849): Loaded HFile hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__ig76rk901v3kinr9f19u92ehel5d04and1kkgrb5ki1m9oenq0a02jbu7ubvdjg8/family1/c8fafc7831e94aabaece01dbc3984833 into bf39f342faf613faca7f43b94cc73bc4/family1 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/family1/a840e944a61f4f37b402c9025eb3b181_SeqId_46_ - updating store file list. 2023-03-05 19:54:51,083 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(882): Loaded HFile hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/family1/a840e944a61f4f37b402c9025eb3b181_SeqId_46_ into bf39f342faf613faca7f43b94cc73bc4/family1 2023-03-05 19:54:51,083 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(855): Successfully loaded hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__ig76rk901v3kinr9f19u92ehel5d04and1kkgrb5ki1m9oenq0a02jbu7ubvdjg8/family1/c8fafc7831e94aabaece01dbc3984833 into bf39f342faf613faca7f43b94cc73bc4/family1 (new location: hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/family1/a840e944a61f4f37b402c9025eb3b181_SeqId_46_) 2023-03-05 19:54:51,083 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__ig76rk901v3kinr9f19u92ehel5d04and1kkgrb5ki1m9oenq0a02jbu7ubvdjg8/family1/c8fafc7831e94aabaece01dbc3984833 2023-03-05 19:54:51,085 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-05 19:54:51,085 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-05 19:54:51,087 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201511309e238e586774429485724a29291fa2b6 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201511309e238e586774429485724a29291fa2b6 2023-03-05 19:54:51,088 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512010e74fc50fe6344c9bf4cc29b51c96a84 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512010e74fc50fe6344c9bf4cc29b51c96a84 2023-03-05 19:54:51,089 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151205d8269d05a54c4de69f2129d205cab5f6 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151205d8269d05a54c4de69f2129d205cab5f6 2023-03-05 19:54:51,089 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 3 expired mob files are deleted 2023-03-05 19:54:51,089 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-03-05 19:54:51,089 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-05 19:54:51,090 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-05 19:54:51,090 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-05 19:54:51,091 DEBUG [PEWorker-3] locking.LockProcedure(242): UNLOCKED pid=42, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-03-05 19:54:51,092 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=42, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE in 1.4060 sec 2023-03-05 19:54:51,167 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.HMaster$12(2454): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly from 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-03-05 19:54:51,168 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure2.ProcedureExecutor(1029): Stored pid=43, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:51,430 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/.tmp/family1/e25a245ba03347debd1913c9d1e7ee76 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/family1/e25a245ba03347debd1913c9d1e7ee76 2023-03-05 19:54:51,435 INFO [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in bf39f342faf613faca7f43b94cc73bc4/family1 of bf39f342faf613faca7f43b94cc73bc4 into e25a245ba03347debd1913c9d1e7ee76(size=6.8 K), total size for store is 390.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-05 19:54:51,435 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for bf39f342faf613faca7f43b94cc73bc4: 2023-03-05 19:54:51,435 INFO [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678046078948.bf39f342faf613faca7f43b94cc73bc4., storeName=family1, fileCount=10, fileSize=50.4 K (5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.6 K), priority=2, time=1678046090587; duration=0sec 2023-03-05 19:54:51,435 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-05 19:54:51,510 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/family1/7b580c5aefad4c738137e91d405ad19a, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/family1/10f3e25324cd4f92875dd59bd77aa3d9, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/family1/ff53d888242a465994b3e3af12d17bbc, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/family1/14ff3ec7787640e0930bc1261c87a834, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/family1/ac2629f2e99c4ee494d00a000a43748b, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/family1/cb5ab59bb2ae41e5bb229de643842ab3, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/family1/058b831671074f5ca1fbc5fc1063227f, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/family1/b1c9923551304bada6fe356e5acfd6b0, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/family1/50f9ccb3289147dc9937be82226b39b2, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/family1/c76304895c3647cb8c7d3b603648ca10_SeqId_44_] to archive 2023-03-05 19:54:51,511 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-05 19:54:51,512 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/family1/7b580c5aefad4c738137e91d405ad19a to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/family1/7b580c5aefad4c738137e91d405ad19a 2023-03-05 19:54:51,514 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/family1/10f3e25324cd4f92875dd59bd77aa3d9 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/family1/10f3e25324cd4f92875dd59bd77aa3d9 2023-03-05 19:54:51,515 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/family1/ff53d888242a465994b3e3af12d17bbc to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/family1/ff53d888242a465994b3e3af12d17bbc 2023-03-05 19:54:51,516 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/family1/14ff3ec7787640e0930bc1261c87a834 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/family1/14ff3ec7787640e0930bc1261c87a834 2023-03-05 19:54:51,517 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/family1/ac2629f2e99c4ee494d00a000a43748b to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/family1/ac2629f2e99c4ee494d00a000a43748b 2023-03-05 19:54:51,518 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/family1/cb5ab59bb2ae41e5bb229de643842ab3 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/family1/cb5ab59bb2ae41e5bb229de643842ab3 2023-03-05 19:54:51,519 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/family1/058b831671074f5ca1fbc5fc1063227f to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/family1/058b831671074f5ca1fbc5fc1063227f 2023-03-05 19:54:51,520 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/family1/b1c9923551304bada6fe356e5acfd6b0 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/family1/b1c9923551304bada6fe356e5acfd6b0 2023-03-05 19:54:51,521 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/family1/50f9ccb3289147dc9937be82226b39b2 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/family1/50f9ccb3289147dc9937be82226b39b2 2023-03-05 19:54:51,523 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/family1/c76304895c3647cb8c7d3b603648ca10_SeqId_44_ to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/family1/c76304895c3647cb8c7d3b603648ca10_SeqId_44_ 2023-03-05 19:54:51,576 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000002 2023-03-05 19:54:51,577 DEBUG [PEWorker-4] util.FSTableDescriptors(519): Deleted hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000001 2023-03-05 19:54:51,577 INFO [PEWorker-4] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000002 2023-03-05 19:54:51,579 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=44, ppid=43, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly}] 2023-03-05 19:54:51,582 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=bf39f342faf613faca7f43b94cc73bc4, REOPEN/MOVE}] 2023-03-05 19:54:51,582 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=bf39f342faf613faca7f43b94cc73bc4, REOPEN/MOVE 2023-03-05 19:54:51,583 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=bf39f342faf613faca7f43b94cc73bc4, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:51,583 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678046078948.bf39f342faf613faca7f43b94cc73bc4.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1678046091583"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678046091583"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678046091583"}]},"ts":"1678046091583"} 2023-03-05 19:54:51,584 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure bf39f342faf613faca7f43b94cc73bc4, server=jenkins-hbase4.apache.org,37037,1678046018983}] 2023-03-05 19:54:51,736 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close bf39f342faf613faca7f43b94cc73bc4 2023-03-05 19:54:51,736 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing bf39f342faf613faca7f43b94cc73bc4, disabling compactions & flushes 2023-03-05 19:54:51,736 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678046078948.bf39f342faf613faca7f43b94cc73bc4. 2023-03-05 19:54:51,736 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678046078948.bf39f342faf613faca7f43b94cc73bc4. 2023-03-05 19:54:51,736 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678046078948.bf39f342faf613faca7f43b94cc73bc4. after waiting 0 ms 2023-03-05 19:54:51,736 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678046078948.bf39f342faf613faca7f43b94cc73bc4. 2023-03-05 19:54:51,740 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/recovered.edits/49.seqid, newMaxSeqId=49, maxSeqId=1 2023-03-05 19:54:51,741 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678046078948.bf39f342faf613faca7f43b94cc73bc4. 2023-03-05 19:54:51,742 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for bf39f342faf613faca7f43b94cc73bc4: 2023-03-05 19:54:51,742 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3509): Not adding moved region record: bf39f342faf613faca7f43b94cc73bc4 to self. 2023-03-05 19:54:51,743 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed bf39f342faf613faca7f43b94cc73bc4 2023-03-05 19:54:51,743 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=bf39f342faf613faca7f43b94cc73bc4, regionState=CLOSED 2023-03-05 19:54:51,743 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678046078948.bf39f342faf613faca7f43b94cc73bc4.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1678046091743"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678046091743"}]},"ts":"1678046091743"} 2023-03-05 19:54:51,746 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=46, resume processing ppid=45 2023-03-05 19:54:51,746 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure bf39f342faf613faca7f43b94cc73bc4, server=jenkins-hbase4.apache.org,37037,1678046018983 in 160 msec 2023-03-05 19:54:51,746 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=bf39f342faf613faca7f43b94cc73bc4, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,37037,1678046018983; forceNewPlan=false, retain=true 2023-03-05 19:54:51,897 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=bf39f342faf613faca7f43b94cc73bc4, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:51,897 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678046078948.bf39f342faf613faca7f43b94cc73bc4.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1678046091897"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678046091897"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678046091897"}]},"ts":"1678046091897"} 2023-03-05 19:54:51,898 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=47, ppid=45, state=RUNNABLE; OpenRegionProcedure bf39f342faf613faca7f43b94cc73bc4, server=jenkins-hbase4.apache.org,37037,1678046018983}] 2023-03-05 19:54:52,053 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678046078948.bf39f342faf613faca7f43b94cc73bc4. 2023-03-05 19:54:52,053 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => bf39f342faf613faca7f43b94cc73bc4, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678046078948.bf39f342faf613faca7f43b94cc73bc4.', STARTKEY => '', ENDKEY => ''} 2023-03-05 19:54:52,053 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly bf39f342faf613faca7f43b94cc73bc4 2023-03-05 19:54:52,053 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678046078948.bf39f342faf613faca7f43b94cc73bc4.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-05 19:54:52,053 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for bf39f342faf613faca7f43b94cc73bc4 2023-03-05 19:54:52,053 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for bf39f342faf613faca7f43b94cc73bc4 2023-03-05 19:54:52,054 INFO [StoreOpener-bf39f342faf613faca7f43b94cc73bc4-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region bf39f342faf613faca7f43b94cc73bc4 2023-03-05 19:54:52,055 DEBUG [StoreOpener-bf39f342faf613faca7f43b94cc73bc4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/family1 2023-03-05 19:54:52,055 DEBUG [StoreOpener-bf39f342faf613faca7f43b94cc73bc4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/family1 2023-03-05 19:54:52,056 INFO [StoreOpener-bf39f342faf613faca7f43b94cc73bc4-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region bf39f342faf613faca7f43b94cc73bc4 columnFamilyName family1 2023-03-05 19:54:52,060 DEBUG [StoreOpener-bf39f342faf613faca7f43b94cc73bc4-1] regionserver.HStore(539): loaded hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/family1/671469e3beed4f10962df0761c25d458 2023-03-05 19:54:52,065 DEBUG [StoreOpener-bf39f342faf613faca7f43b94cc73bc4-1] regionserver.HStore(539): loaded hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/family1/a840e944a61f4f37b402c9025eb3b181_SeqId_46_ 2023-03-05 19:54:52,068 DEBUG [StoreOpener-bf39f342faf613faca7f43b94cc73bc4-1] regionserver.HStore(539): loaded hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/family1/e25a245ba03347debd1913c9d1e7ee76 2023-03-05 19:54:52,068 INFO [StoreOpener-bf39f342faf613faca7f43b94cc73bc4-1] regionserver.HStore(310): Store=bf39f342faf613faca7f43b94cc73bc4/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-05 19:54:52,069 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4 2023-03-05 19:54:52,070 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4 2023-03-05 19:54:52,072 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for bf39f342faf613faca7f43b94cc73bc4 2023-03-05 19:54:52,072 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened bf39f342faf613faca7f43b94cc73bc4; next sequenceid=50; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73219243, jitterRate=0.09105174243450165}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-05 19:54:52,072 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for bf39f342faf613faca7f43b94cc73bc4: 2023-03-05 19:54:52,073 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678046078948.bf39f342faf613faca7f43b94cc73bc4., pid=47, masterSystemTime=1678046092050 2023-03-05 19:54:52,073 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-05 19:54:52,074 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-03-05 19:54:52,075 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 399433 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-03-05 19:54:52,075 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.HStore(1912): bf39f342faf613faca7f43b94cc73bc4/family1 is initiating minor compaction (all files) 2023-03-05 19:54:52,075 INFO [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of bf39f342faf613faca7f43b94cc73bc4/family1 in testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678046078948.bf39f342faf613faca7f43b94cc73bc4. 2023-03-05 19:54:52,075 INFO [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/family1/671469e3beed4f10962df0761c25d458, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/family1/e25a245ba03347debd1913c9d1e7ee76, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/family1/a840e944a61f4f37b402c9025eb3b181_SeqId_46_] into tmpdir=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/.tmp, totalSize=390.1 K 2023-03-05 19:54:52,075 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678046078948.bf39f342faf613faca7f43b94cc73bc4. 2023-03-05 19:54:52,075 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678046078948.bf39f342faf613faca7f43b94cc73bc4. 2023-03-05 19:54:52,075 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.Compactor(207): Compacting 671469e3beed4f10962df0761c25d458, keycount=1000, bloomtype=ROW, size=191.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878843274 2023-03-05 19:54:52,076 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=bf39f342faf613faca7f43b94cc73bc4, regionState=OPEN, openSeqNum=50, regionLocation=jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:52,076 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678046078948.bf39f342faf613faca7f43b94cc73bc4.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1678046092075"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678046092075"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678046092075"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678046092075"}]},"ts":"1678046092075"} 2023-03-05 19:54:52,076 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.Compactor(207): Compacting e25a245ba03347debd1913c9d1e7ee76, keycount=9, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621243274 2023-03-05 19:54:52,076 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.Compactor(207): Compacting a840e944a61f4f37b402c9025eb3b181_SeqId_46_, keycount=1002, bloomtype=ROW, size=191.8 K, encoding=NONE, compression=NONE, seqNum=47, earliestPutTs=1448878843274 2023-03-05 19:54:52,079 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=47, resume processing ppid=45 2023-03-05 19:54:52,079 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=47, ppid=45, state=SUCCESS; OpenRegionProcedure bf39f342faf613faca7f43b94cc73bc4, server=jenkins-hbase4.apache.org,37037,1678046018983 in 179 msec 2023-03-05 19:54:52,080 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=45, resume processing ppid=44 2023-03-05 19:54:52,080 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=bf39f342faf613faca7f43b94cc73bc4, REOPEN/MOVE in 498 msec 2023-03-05 19:54:52,083 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=44, resume processing ppid=43 2023-03-05 19:54:52,083 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=44, ppid=43, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 502 msec 2023-03-05 19:54:52,084 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=43, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 916 msec 2023-03-05 19:54:52,085 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1227): Checking to see if procedure is done pid=43 2023-03-05 19:54:52,087 INFO [Listener at localhost/40787] compactions.TestMobCompactor(278): alter status finished 2023-03-05 19:54:52,087 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=38221] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:52,089 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] procedure2.ProcedureExecutor(1029): Stored pid=48, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-03-05 19:54:52,095 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=48, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-03-05 19:54:52,096 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-05 19:54:52,096 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=48, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678046692096 2023-03-05 19:54:52,097 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-05 19:54:52,097 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 5 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column 'family1' 2023-03-05 19:54:52,097 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-05 19:54:52,098 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] zookeeper.ReadOnlyZKClient(139): Connect 0x0a90ad22 to 127.0.0.1:62957 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-05 19:54:52,101 INFO [RS:0;jenkins-hbase4:37037-shortCompactions-0] throttle.PressureAwareThroughputController(145): bf39f342faf613faca7f43b94cc73bc4#family1#compaction#61 average throughput is 10.03 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-05 19:54:52,112 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@29f4d695, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-05 19:54:52,113 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-05 19:54:52,113 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-05 19:54:52,113 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-05 19:54:52,113 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-05 19:54:52,113 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-05 19:54:52,937 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/.tmp/family1/60640196f1834ea09f73f7e576bcece7 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/family1/60640196f1834ea09f73f7e576bcece7 2023-03-05 19:54:52,943 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128016235daa0f643dba975df86e7a56223 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151128016235daa0f643dba975df86e7a56223 2023-03-05 19:54:52,943 INFO [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in bf39f342faf613faca7f43b94cc73bc4/family1 of bf39f342faf613faca7f43b94cc73bc4 into 60640196f1834ea09f73f7e576bcece7(size=193.0 K), total size for store is 193.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-05 19:54:52,943 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for bf39f342faf613faca7f43b94cc73bc4: 2023-03-05 19:54:52,943 INFO [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678046078948.bf39f342faf613faca7f43b94cc73bc4., storeName=bf39f342faf613faca7f43b94cc73bc4/family1, priority=13, startTime=1678046092073; duration=0sec 2023-03-05 19:54:52,944 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-05 19:54:52,944 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-05 19:54:52,957 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-05 19:54:52,958 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:48024, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-05 19:54:52,964 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-05 19:54:52,966 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:51096, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-05 19:54:52,974 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/81fe10ba282e473dba983444d86d1d9d first=Optional[r05] last=Optional[r8] 2023-03-05 19:54:52,974 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678046078948.bf39f342faf613faca7f43b94cc73bc4., hostname=jenkins-hbase4.apache.org,37037,1678046018983, seqNum=50 for row with hfile group [{family1,hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/81fe10ba282e473dba983444d86d1d9d}] 2023-03-05 19:54:52,978 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(763): Validating hfile at hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/81fe10ba282e473dba983444d86d1d9d for inclusion in bf39f342faf613faca7f43b94cc73bc4/family1 2023-03-05 19:54:52,980 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(775): HFile bounds: first=r05 last=r8 2023-03-05 19:54:52,980 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(777): Region bounds: first= last= 2023-03-05 19:54:52,981 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HRegion(2446): Flush status journal for bf39f342faf613faca7f43b94cc73bc4: 2023-03-05 19:54:52,981 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/81fe10ba282e473dba983444d86d1d9d to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__sk6a400jr12k4rsmcu7imvvjp30h70hedac3mdrisf4hnnrbohlk5cmslplrtutf/family1/81fe10ba282e473dba983444d86d1d9d 2023-03-05 19:54:52,983 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__sk6a400jr12k4rsmcu7imvvjp30h70hedac3mdrisf4hnnrbohlk5cmslplrtutf/family1/81fe10ba282e473dba983444d86d1d9d as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/family1/28eb515dc79345d993adf5960966d725_SeqId_53_ 2023-03-05 19:54:52,984 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(849): Loaded HFile hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__sk6a400jr12k4rsmcu7imvvjp30h70hedac3mdrisf4hnnrbohlk5cmslplrtutf/family1/81fe10ba282e473dba983444d86d1d9d into bf39f342faf613faca7f43b94cc73bc4/family1 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/family1/28eb515dc79345d993adf5960966d725_SeqId_53_ - updating store file list. 2023-03-05 19:54:52,987 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(882): Loaded HFile hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/family1/28eb515dc79345d993adf5960966d725_SeqId_53_ into bf39f342faf613faca7f43b94cc73bc4/family1 2023-03-05 19:54:52,987 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(855): Successfully loaded hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__sk6a400jr12k4rsmcu7imvvjp30h70hedac3mdrisf4hnnrbohlk5cmslplrtutf/family1/81fe10ba282e473dba983444d86d1d9d into bf39f342faf613faca7f43b94cc73bc4/family1 (new location: hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/family1/28eb515dc79345d993adf5960966d725_SeqId_53_) 2023-03-05 19:54:52,988 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__sk6a400jr12k4rsmcu7imvvjp30h70hedac3mdrisf4hnnrbohlk5cmslplrtutf/family1/81fe10ba282e473dba983444d86d1d9d 2023-03-05 19:54:52,990 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-05 19:54:52,991 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-05 19:54:52,992 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015112019abd20e29104d17badd7082788f293b to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015112019abd20e29104d17badd7082788f293b 2023-03-05 19:54:52,993 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015112832c982860ed24b92943729ef102f47a5 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015112832c982860ed24b92943729ef102f47a5 2023-03-05 19:54:52,993 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2 expired mob files are deleted 2023-03-05 19:54:52,993 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-03-05 19:54:52,993 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-05 19:54:52,993 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-05 19:54:52,993 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-05 19:54:52,993 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-05 19:54:52,993 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-05 19:54:52,994 DEBUG [PEWorker-5] locking.LockProcedure(242): UNLOCKED pid=48, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-03-05 19:54:52,996 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=48, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE in 906 msec 2023-03-05 19:54:53,000 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/family1/671469e3beed4f10962df0761c25d458, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/family1/e25a245ba03347debd1913c9d1e7ee76, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/family1/a840e944a61f4f37b402c9025eb3b181_SeqId_46_] to archive 2023-03-05 19:54:53,000 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-05 19:54:53,001 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/family1/671469e3beed4f10962df0761c25d458 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/family1/671469e3beed4f10962df0761c25d458 2023-03-05 19:54:53,002 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/family1/e25a245ba03347debd1913c9d1e7ee76 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/family1/e25a245ba03347debd1913c9d1e7ee76 2023-03-05 19:54:53,003 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/family1/a840e944a61f4f37b402c9025eb3b181_SeqId_46_ to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/family1/a840e944a61f4f37b402c9025eb3b181_SeqId_46_ 2023-03-05 19:54:53,045 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.HMaster$12(2454): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly from 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-03-05 19:54:53,046 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure2.ProcedureExecutor(1029): Stored pid=49, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:53,455 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000003 2023-03-05 19:54:53,456 DEBUG [PEWorker-3] util.FSTableDescriptors(519): Deleted hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000002 2023-03-05 19:54:53,456 INFO [PEWorker-3] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000003 2023-03-05 19:54:53,458 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=50, ppid=49, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly}] 2023-03-05 19:54:53,461 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=51, ppid=50, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=bf39f342faf613faca7f43b94cc73bc4, REOPEN/MOVE}] 2023-03-05 19:54:53,461 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=51, ppid=50, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=bf39f342faf613faca7f43b94cc73bc4, REOPEN/MOVE 2023-03-05 19:54:53,462 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=bf39f342faf613faca7f43b94cc73bc4, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:53,462 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678046078948.bf39f342faf613faca7f43b94cc73bc4.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1678046093462"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678046093462"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678046093462"}]},"ts":"1678046093462"} 2023-03-05 19:54:53,463 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=52, ppid=51, state=RUNNABLE; CloseRegionProcedure bf39f342faf613faca7f43b94cc73bc4, server=jenkins-hbase4.apache.org,37037,1678046018983}] 2023-03-05 19:54:53,615 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close bf39f342faf613faca7f43b94cc73bc4 2023-03-05 19:54:53,615 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing bf39f342faf613faca7f43b94cc73bc4, disabling compactions & flushes 2023-03-05 19:54:53,615 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678046078948.bf39f342faf613faca7f43b94cc73bc4. 2023-03-05 19:54:53,615 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678046078948.bf39f342faf613faca7f43b94cc73bc4. 2023-03-05 19:54:53,615 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678046078948.bf39f342faf613faca7f43b94cc73bc4. after waiting 0 ms 2023-03-05 19:54:53,615 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678046078948.bf39f342faf613faca7f43b94cc73bc4. 2023-03-05 19:54:53,621 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/recovered.edits/55.seqid, newMaxSeqId=55, maxSeqId=49 2023-03-05 19:54:53,622 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678046078948.bf39f342faf613faca7f43b94cc73bc4. 2023-03-05 19:54:53,622 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for bf39f342faf613faca7f43b94cc73bc4: 2023-03-05 19:54:53,622 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3509): Not adding moved region record: bf39f342faf613faca7f43b94cc73bc4 to self. 2023-03-05 19:54:53,623 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed bf39f342faf613faca7f43b94cc73bc4 2023-03-05 19:54:53,624 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=bf39f342faf613faca7f43b94cc73bc4, regionState=CLOSED 2023-03-05 19:54:53,624 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678046078948.bf39f342faf613faca7f43b94cc73bc4.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1678046093624"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678046093624"}]},"ts":"1678046093624"} 2023-03-05 19:54:53,626 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=52, resume processing ppid=51 2023-03-05 19:54:53,626 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=52, ppid=51, state=SUCCESS; CloseRegionProcedure bf39f342faf613faca7f43b94cc73bc4, server=jenkins-hbase4.apache.org,37037,1678046018983 in 162 msec 2023-03-05 19:54:53,627 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=51, ppid=50, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=bf39f342faf613faca7f43b94cc73bc4, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,37037,1678046018983; forceNewPlan=false, retain=true 2023-03-05 19:54:53,777 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=bf39f342faf613faca7f43b94cc73bc4, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:53,777 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678046078948.bf39f342faf613faca7f43b94cc73bc4.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1678046093777"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678046093777"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678046093777"}]},"ts":"1678046093777"} 2023-03-05 19:54:53,779 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=53, ppid=51, state=RUNNABLE; OpenRegionProcedure bf39f342faf613faca7f43b94cc73bc4, server=jenkins-hbase4.apache.org,37037,1678046018983}] 2023-03-05 19:54:53,933 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678046078948.bf39f342faf613faca7f43b94cc73bc4. 2023-03-05 19:54:53,933 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => bf39f342faf613faca7f43b94cc73bc4, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678046078948.bf39f342faf613faca7f43b94cc73bc4.', STARTKEY => '', ENDKEY => ''} 2023-03-05 19:54:53,934 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly bf39f342faf613faca7f43b94cc73bc4 2023-03-05 19:54:53,934 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678046078948.bf39f342faf613faca7f43b94cc73bc4.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-05 19:54:53,934 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for bf39f342faf613faca7f43b94cc73bc4 2023-03-05 19:54:53,934 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for bf39f342faf613faca7f43b94cc73bc4 2023-03-05 19:54:53,935 INFO [StoreOpener-bf39f342faf613faca7f43b94cc73bc4-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region bf39f342faf613faca7f43b94cc73bc4 2023-03-05 19:54:53,936 DEBUG [StoreOpener-bf39f342faf613faca7f43b94cc73bc4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/family1 2023-03-05 19:54:53,936 DEBUG [StoreOpener-bf39f342faf613faca7f43b94cc73bc4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/family1 2023-03-05 19:54:53,937 INFO [StoreOpener-bf39f342faf613faca7f43b94cc73bc4-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region bf39f342faf613faca7f43b94cc73bc4 columnFamilyName family1 2023-03-05 19:54:53,943 DEBUG [StoreOpener-bf39f342faf613faca7f43b94cc73bc4-1] regionserver.HStore(539): loaded hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/family1/28eb515dc79345d993adf5960966d725_SeqId_53_ 2023-03-05 19:54:53,946 DEBUG [StoreOpener-bf39f342faf613faca7f43b94cc73bc4-1] regionserver.HStore(539): loaded hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/family1/60640196f1834ea09f73f7e576bcece7 2023-03-05 19:54:53,946 INFO [StoreOpener-bf39f342faf613faca7f43b94cc73bc4-1] regionserver.HStore(310): Store=bf39f342faf613faca7f43b94cc73bc4/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-05 19:54:53,947 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4 2023-03-05 19:54:53,949 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4 2023-03-05 19:54:53,951 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for bf39f342faf613faca7f43b94cc73bc4 2023-03-05 19:54:53,952 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened bf39f342faf613faca7f43b94cc73bc4; next sequenceid=56; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70374140, jitterRate=0.0486564040184021}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-05 19:54:53,952 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for bf39f342faf613faca7f43b94cc73bc4: 2023-03-05 19:54:53,952 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678046078948.bf39f342faf613faca7f43b94cc73bc4., pid=53, masterSystemTime=1678046093930 2023-03-05 19:54:53,954 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678046078948.bf39f342faf613faca7f43b94cc73bc4. 2023-03-05 19:54:53,954 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678046078948.bf39f342faf613faca7f43b94cc73bc4. 2023-03-05 19:54:53,954 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=bf39f342faf613faca7f43b94cc73bc4, regionState=OPEN, openSeqNum=56, regionLocation=jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:53,954 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678046078948.bf39f342faf613faca7f43b94cc73bc4.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1678046093954"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678046093954"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678046093954"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678046093954"}]},"ts":"1678046093954"} 2023-03-05 19:54:53,957 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=53, resume processing ppid=51 2023-03-05 19:54:53,958 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=53, ppid=51, state=SUCCESS; OpenRegionProcedure bf39f342faf613faca7f43b94cc73bc4, server=jenkins-hbase4.apache.org,37037,1678046018983 in 177 msec 2023-03-05 19:54:53,959 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=51, resume processing ppid=50 2023-03-05 19:54:53,959 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=51, ppid=50, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=bf39f342faf613faca7f43b94cc73bc4, REOPEN/MOVE in 497 msec 2023-03-05 19:54:53,961 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=50, resume processing ppid=49 2023-03-05 19:54:53,961 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=50, ppid=49, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 502 msec 2023-03-05 19:54:53,963 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=49, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 917 msec 2023-03-05 19:54:53,964 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1227): Checking to see if procedure is done pid=49 2023-03-05 19:54:53,965 INFO [Listener at localhost/40787] compactions.TestMobCompactor(278): alter status finished 2023-03-05 19:54:53,966 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=38221] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-05 19:54:53,967 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] procedure2.ProcedureExecutor(1029): Stored pid=54, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-03-05 19:54:53,968 DEBUG [PEWorker-1] locking.LockProcedure(309): LOCKED pid=54, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-03-05 19:54:53,969 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-05 19:54:53,969 INFO [PEWorker-1] procedure2.TimeoutExecutorThread(81): ADDED pid=54, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678046693969 2023-03-05 19:54:53,970 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-05 19:54:53,970 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 4 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column 'family1' 2023-03-05 19:54:53,970 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-05 19:54:53,971 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] zookeeper.ReadOnlyZKClient(139): Connect 0x0fff9741 to 127.0.0.1:62957 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-05 19:54:53,975 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@30a855, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-05 19:54:53,976 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-05 19:54:53,976 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-05 19:54:53,976 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-05 19:54:53,976 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-05 19:54:53,976 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-05 19:54:53,976 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-05 19:54:53,976 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-05 19:54:53,976 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-05 19:54:53,976 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-05 19:54:53,976 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-05 19:54:53,979 DEBUG [PEWorker-2] locking.LockProcedure(242): UNLOCKED pid=54, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-03-05 19:54:53,980 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=54, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE in 12 msec 2023-03-05 19:54:54,028 INFO [Listener at localhost/40787] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly Thread=248 (was 256), OpenFileDescriptor=624 (was 618) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=195 (was 231), ProcessCount=173 (was 173), AvailableMemoryMB=4736 (was 4839) 2023-03-05 19:54:54,040 INFO [Listener at localhost/40787] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily Thread=248, OpenFileDescriptor=624, MaxFileDescriptor=60000, SystemLoadAverage=195, ProcessCount=173, AvailableMemoryMB=4736 2023-03-05 19:54:54,042 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-03-05 19:54:54,043 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure2.ProcedureExecutor(1029): Stored pid=55, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:54,044 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_PRE_OPERATION 2023-03-05 19:54:54,044 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily" procId is: 55 2023-03-05 19:54:54,045 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-05 19:54:54,045 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-03-05 19:54:54,046 DEBUG [HFileArchiver-10] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb 2023-03-05 19:54:54,047 DEBUG [HFileArchiver-10] backup.HFileArchiver(153): Directory hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb empty. 2023-03-05 19:54:54,047 DEBUG [HFileArchiver-10] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb 2023-03-05 19:54:54,047 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily regions 2023-03-05 19:54:54,146 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-03-05 19:54:54,347 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-03-05 19:54:54,455 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000001 2023-03-05 19:54:54,457 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(7675): creating {ENCODED => b9c878f961e0a32027f8960cfe26b4eb, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678046094042.b9c878f961e0a32027f8960cfe26b4eb.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/.tmp 2023-03-05 19:54:54,464 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678046094042.b9c878f961e0a32027f8960cfe26b4eb.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-05 19:54:54,464 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1604): Closing b9c878f961e0a32027f8960cfe26b4eb, disabling compactions & flushes 2023-03-05 19:54:54,464 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678046094042.b9c878f961e0a32027f8960cfe26b4eb. 2023-03-05 19:54:54,464 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678046094042.b9c878f961e0a32027f8960cfe26b4eb. 2023-03-05 19:54:54,464 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678046094042.b9c878f961e0a32027f8960cfe26b4eb. after waiting 0 ms 2023-03-05 19:54:54,464 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678046094042.b9c878f961e0a32027f8960cfe26b4eb. 2023-03-05 19:54:54,464 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678046094042.b9c878f961e0a32027f8960cfe26b4eb. 2023-03-05 19:54:54,464 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1558): Region close journal for b9c878f961e0a32027f8960cfe26b4eb: 2023-03-05 19:54:54,466 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_ADD_TO_META 2023-03-05 19:54:54,467 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678046094042.b9c878f961e0a32027f8960cfe26b4eb.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1678046094467"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678046094467"}]},"ts":"1678046094467"} 2023-03-05 19:54:54,468 INFO [PEWorker-5] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-03-05 19:54:54,469 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-05 19:54:54,469 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678046094469"}]},"ts":"1678046094469"} 2023-03-05 19:54:54,470 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, state=ENABLING in hbase:meta 2023-03-05 19:54:54,473 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=56, ppid=55, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=b9c878f961e0a32027f8960cfe26b4eb, ASSIGN}] 2023-03-05 19:54:54,475 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, ppid=55, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=b9c878f961e0a32027f8960cfe26b4eb, ASSIGN 2023-03-05 19:54:54,475 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=56, ppid=55, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=b9c878f961e0a32027f8960cfe26b4eb, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,37037,1678046018983; forceNewPlan=false, retain=false 2023-03-05 19:54:54,626 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=56 updating hbase:meta row=b9c878f961e0a32027f8960cfe26b4eb, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:54,626 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678046094042.b9c878f961e0a32027f8960cfe26b4eb.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1678046094626"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678046094626"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678046094626"}]},"ts":"1678046094626"} 2023-03-05 19:54:54,628 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=57, ppid=56, state=RUNNABLE; OpenRegionProcedure b9c878f961e0a32027f8960cfe26b4eb, server=jenkins-hbase4.apache.org,37037,1678046018983}] 2023-03-05 19:54:54,648 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-03-05 19:54:54,782 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678046094042.b9c878f961e0a32027f8960cfe26b4eb. 2023-03-05 19:54:54,782 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => b9c878f961e0a32027f8960cfe26b4eb, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678046094042.b9c878f961e0a32027f8960cfe26b4eb.', STARTKEY => '', ENDKEY => ''} 2023-03-05 19:54:54,782 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily b9c878f961e0a32027f8960cfe26b4eb 2023-03-05 19:54:54,783 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678046094042.b9c878f961e0a32027f8960cfe26b4eb.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-05 19:54:54,783 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for b9c878f961e0a32027f8960cfe26b4eb 2023-03-05 19:54:54,783 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for b9c878f961e0a32027f8960cfe26b4eb 2023-03-05 19:54:54,784 INFO [StoreOpener-b9c878f961e0a32027f8960cfe26b4eb-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region b9c878f961e0a32027f8960cfe26b4eb 2023-03-05 19:54:54,785 DEBUG [StoreOpener-b9c878f961e0a32027f8960cfe26b4eb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/family1 2023-03-05 19:54:54,785 DEBUG [StoreOpener-b9c878f961e0a32027f8960cfe26b4eb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/family1 2023-03-05 19:54:54,785 INFO [StoreOpener-b9c878f961e0a32027f8960cfe26b4eb-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b9c878f961e0a32027f8960cfe26b4eb columnFamilyName family1 2023-03-05 19:54:54,787 INFO [StoreOpener-b9c878f961e0a32027f8960cfe26b4eb-1] regionserver.HStore(310): Store=b9c878f961e0a32027f8960cfe26b4eb/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-05 19:54:54,787 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb 2023-03-05 19:54:54,788 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb 2023-03-05 19:54:54,790 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for b9c878f961e0a32027f8960cfe26b4eb 2023-03-05 19:54:54,792 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-05 19:54:54,792 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened b9c878f961e0a32027f8960cfe26b4eb; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69704965, jitterRate=0.038684919476509094}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-05 19:54:54,792 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for b9c878f961e0a32027f8960cfe26b4eb: 2023-03-05 19:54:54,793 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678046094042.b9c878f961e0a32027f8960cfe26b4eb., pid=57, masterSystemTime=1678046094779 2023-03-05 19:54:54,794 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678046094042.b9c878f961e0a32027f8960cfe26b4eb. 2023-03-05 19:54:54,794 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678046094042.b9c878f961e0a32027f8960cfe26b4eb. 2023-03-05 19:54:54,794 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=56 updating hbase:meta row=b9c878f961e0a32027f8960cfe26b4eb, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:54,794 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678046094042.b9c878f961e0a32027f8960cfe26b4eb.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1678046094794"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678046094794"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678046094794"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678046094794"}]},"ts":"1678046094794"} 2023-03-05 19:54:54,797 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=57, resume processing ppid=56 2023-03-05 19:54:54,797 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=57, ppid=56, state=SUCCESS; OpenRegionProcedure b9c878f961e0a32027f8960cfe26b4eb, server=jenkins-hbase4.apache.org,37037,1678046018983 in 168 msec 2023-03-05 19:54:54,798 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=56, resume processing ppid=55 2023-03-05 19:54:54,798 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=56, ppid=55, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=b9c878f961e0a32027f8960cfe26b4eb, ASSIGN in 324 msec 2023-03-05 19:54:54,799 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-05 19:54:54,799 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678046094799"}]},"ts":"1678046094799"} 2023-03-05 19:54:54,800 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, state=ENABLED in hbase:meta 2023-03-05 19:54:54,802 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_POST_OPERATION 2023-03-05 19:54:54,803 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=55, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 760 msec 2023-03-05 19:54:55,149 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-03-05 19:54:55,150 INFO [Listener at localhost/40787] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, procId: 55 completed 2023-03-05 19:54:55,169 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-05 19:54:55,171 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:55,171 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-05 19:54:55,171 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:54:55,171 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-05 19:54:55,171 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-05 19:54:55,171 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:55,171 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:55,173 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:54:55,173 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:55,173 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:54:55,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:54:55,173 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:55,173 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-05 19:54:55,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:55,173 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:55,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-05 19:54:55,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:55,174 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:55,174 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:55,174 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-05 19:54:55,174 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:54:55,174 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-05 19:54:55,174 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-05 19:54:55,174 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-05 19:54:55,174 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678046094042.b9c878f961e0a32027f8960cfe26b4eb. 2023-03-05 19:54:55,175 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678046094042.b9c878f961e0a32027f8960cfe26b4eb. started... 2023-03-05 19:54:55,175 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing b9c878f961e0a32027f8960cfe26b4eb 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-03-05 19:54:55,599 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130f50f2d703f324c25aa44937e76337f51 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151130f50f2d703f324c25aa44937e76337f51 2023-03-05 19:54:56,003 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/.tmp/family1/a1cf7b892fdf49569eaa81f22bfaf698 2023-03-05 19:54:56,007 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/.tmp/family1/a1cf7b892fdf49569eaa81f22bfaf698 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/family1/a1cf7b892fdf49569eaa81f22bfaf698 2023-03-05 19:54:56,011 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/family1/a1cf7b892fdf49569eaa81f22bfaf698, entries=1000, sequenceid=5, filesize=190.5 K 2023-03-05 19:54:56,012 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for b9c878f961e0a32027f8960cfe26b4eb in 836ms, sequenceid=5, compaction requested=false 2023-03-05 19:54:56,012 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-05 19:54:56,012 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for b9c878f961e0a32027f8960cfe26b4eb: 2023-03-05 19:54:56,013 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678046094042.b9c878f961e0a32027f8960cfe26b4eb. 2023-03-05 19:54:56,013 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-05 19:54:56,013 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-05 19:54:56,013 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:56,013 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-05 19:54:56,013 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-05 19:54:56,014 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:56,014 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:56,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:56,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:56,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:56,015 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:56,015 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-05 19:54:56,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:56,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:56,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:56,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:56,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:56,016 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-05 19:54:56,016 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@41d7b202[Count = 0] remaining members to acquire global barrier 2023-03-05 19:54:56,016 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-05 19:54:56,016 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:56,017 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:56,017 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:56,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:56,017 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-05 19:54:56,017 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-05 19:54:56,017 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,37037,1678046018983' in zk 2023-03-05 19:54:56,017 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:56,017 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-05 19:54:56,019 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-05 19:54:56,019 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:56,019 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:54:56,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:56,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:56,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:56,020 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-05 19:54:56,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:56,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:56,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:56,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:56,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:56,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:56,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:56,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,37037,1678046018983': 2023-03-05 19:54:56,022 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-05 19:54:56,022 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-05 19:54:56,022 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-05 19:54:56,022 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-05 19:54:56,022 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:56,022 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-05 19:54:56,023 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:56,023 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:56,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:56,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:56,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:56,023 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:54:56,023 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:56,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:56,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:56,023 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:54:56,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:54:56,023 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:56,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:56,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:56,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:56,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:56,024 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:56,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:56,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:56,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:56,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:56,029 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:56,029 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:54:56,029 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:56,030 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:54:56,030 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:56,030 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:54:56,029 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:54:56,030 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-05 19:54:56,030 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-05 19:54:56,030 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:56,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:54:56,030 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:56,030 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:56,030 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:56,030 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-05 19:54:56,030 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-05 19:54:56,030 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:54:56,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:54:56,130 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-05 19:54:56,131 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-05 19:54:56,135 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-05 19:54:56,136 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-03-05 19:54:56,136 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:56,136 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-05 19:54:56,137 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:54:56,137 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-05 19:54:56,137 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-05 19:54:56,137 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:56,137 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:56,138 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:54:56,138 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:56,138 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:54:56,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:54:56,139 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:56,139 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-05 19:54:56,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:56,139 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:56,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-05 19:54:56,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:56,139 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:56,139 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-03-05 19:54:56,139 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:56,139 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-05 19:54:56,140 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:54:56,140 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-05 19:54:56,140 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-05 19:54:56,140 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-05 19:54:56,140 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678046094042.b9c878f961e0a32027f8960cfe26b4eb. 2023-03-05 19:54:56,140 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678046094042.b9c878f961e0a32027f8960cfe26b4eb. started... 2023-03-05 19:54:56,140 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing b9c878f961e0a32027f8960cfe26b4eb 1/1 column families, dataSize=74 B heapSize=408 B 2023-03-05 19:54:56,552 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511288cf82d5dd37a492784e0be184bdce971 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201511288cf82d5dd37a492784e0be184bdce971 2023-03-05 19:54:56,796 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x0fff9741 to 127.0.0.1:62957 2023-03-05 19:54:56,796 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-05 19:54:56,797 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-05 19:54:56,797 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x507c8885 to 127.0.0.1:62957 2023-03-05 19:54:56,797 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-05 19:54:56,797 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-05 19:54:56,797 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x1f3acba3 to 127.0.0.1:62957 2023-03-05 19:54:56,797 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-05 19:54:56,798 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-05 19:54:56,798 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x58345cef to 127.0.0.1:62957 2023-03-05 19:54:56,798 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-05 19:54:56,798 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-05 19:54:56,798 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x0a90ad22 to 127.0.0.1:62957 2023-03-05 19:54:56,798 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-05 19:54:56,957 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/.tmp/family1/77334e1802b241ab971852cbeb33bac5 2023-03-05 19:54:56,962 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/.tmp/family1/77334e1802b241ab971852cbeb33bac5 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/family1/77334e1802b241ab971852cbeb33bac5 2023-03-05 19:54:56,965 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/family1/77334e1802b241ab971852cbeb33bac5, entries=1, sequenceid=9, filesize=5.0 K 2023-03-05 19:54:56,966 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for b9c878f961e0a32027f8960cfe26b4eb in 826ms, sequenceid=9, compaction requested=false 2023-03-05 19:54:56,966 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for b9c878f961e0a32027f8960cfe26b4eb: 2023-03-05 19:54:56,966 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678046094042.b9c878f961e0a32027f8960cfe26b4eb. 2023-03-05 19:54:56,966 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-05 19:54:56,966 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-05 19:54:56,966 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:56,966 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-05 19:54:56,966 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-05 19:54:56,968 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:56,968 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:56,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:56,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:56,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:56,968 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:56,968 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-05 19:54:56,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:56,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:56,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:56,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:56,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:56,969 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-05 19:54:56,969 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@244660b7[Count = 0] remaining members to acquire global barrier 2023-03-05 19:54:56,969 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-05 19:54:56,969 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:56,970 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:56,970 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:56,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:56,971 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-05 19:54:56,971 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-05 19:54:56,971 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,37037,1678046018983' in zk 2023-03-05 19:54:56,971 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:56,971 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-05 19:54:56,972 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:56,972 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-05 19:54:56,972 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:54:56,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:56,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:56,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:56,972 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-05 19:54:56,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:56,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:56,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:56,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:56,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:56,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:56,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:56,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,37037,1678046018983': 2023-03-05 19:54:56,974 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-05 19:54:56,974 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-05 19:54:56,974 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-05 19:54:56,974 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-05 19:54:56,974 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:56,974 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-05 19:54:56,976 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:56,976 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:56,976 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:56,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:56,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:56,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:56,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:56,976 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:54:56,977 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:54:56,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:56,977 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:56,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:54:56,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:56,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:56,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:56,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:56,978 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:56,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:56,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:56,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:56,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:56,980 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:56,980 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:54:56,980 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:56,980 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:54:56,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:54:56,980 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:54:56,980 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-05 19:54:56,980 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-05 19:54:56,980 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:54:56,980 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:56,980 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:54:56,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:54:56,980 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:56,980 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-03-05 19:54:56,980 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:56,980 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-05 19:54:56,980 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-05 19:54:56,980 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:56,980 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:57,081 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-05 19:54:57,081 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-05 19:54:57,085 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-05 19:54:57,086 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-03-05 19:54:57,087 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:57,087 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-05 19:54:57,087 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:54:57,087 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-05 19:54:57,087 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-05 19:54:57,087 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:57,087 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:57,088 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:54:57,088 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:57,088 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:54:57,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:54:57,089 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:57,089 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-05 19:54:57,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:57,089 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:57,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-05 19:54:57,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:57,089 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:57,090 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-03-05 19:54:57,090 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:57,090 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-05 19:54:57,090 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:54:57,090 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-05 19:54:57,090 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-05 19:54:57,090 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-05 19:54:57,090 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678046094042.b9c878f961e0a32027f8960cfe26b4eb. 2023-03-05 19:54:57,090 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678046094042.b9c878f961e0a32027f8960cfe26b4eb. started... 2023-03-05 19:54:57,090 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing b9c878f961e0a32027f8960cfe26b4eb 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-05 19:54:57,304 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-05 19:54:57,504 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151201cf1b200636fe4a06b6f23e8335b543a6 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151201cf1b200636fe4a06b6f23e8335b543a6 2023-03-05 19:54:57,910 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/.tmp/family1/0701549599d349ffada783dd7788b746 2023-03-05 19:54:57,914 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/.tmp/family1/0701549599d349ffada783dd7788b746 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/family1/0701549599d349ffada783dd7788b746 2023-03-05 19:54:57,917 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/family1/0701549599d349ffada783dd7788b746, entries=1, sequenceid=13, filesize=5.0 K 2023-03-05 19:54:57,918 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for b9c878f961e0a32027f8960cfe26b4eb in 828ms, sequenceid=13, compaction requested=true 2023-03-05 19:54:57,918 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for b9c878f961e0a32027f8960cfe26b4eb: 2023-03-05 19:54:57,918 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678046094042.b9c878f961e0a32027f8960cfe26b4eb. 2023-03-05 19:54:57,918 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-05 19:54:57,918 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-05 19:54:57,918 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:57,918 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-05 19:54:57,918 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-05 19:54:57,920 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:57,920 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:57,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:57,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:57,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:57,921 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:57,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:57,921 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-05 19:54:57,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:57,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:57,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:57,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:57,922 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-05 19:54:57,922 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1ebe2001[Count = 0] remaining members to acquire global barrier 2023-03-05 19:54:57,922 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-05 19:54:57,922 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:57,924 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:57,924 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:57,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:57,924 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-05 19:54:57,924 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-05 19:54:57,924 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:57,924 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-05 19:54:57,924 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,37037,1678046018983' in zk 2023-03-05 19:54:57,925 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-05 19:54:57,925 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:54:57,925 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:57,925 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-05 19:54:57,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:57,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:57,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:57,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:57,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:57,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:57,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:57,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:57,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:57,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:57,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,37037,1678046018983': 2023-03-05 19:54:57,927 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-05 19:54:57,927 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-05 19:54:57,927 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-05 19:54:57,927 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-05 19:54:57,927 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:57,927 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-05 19:54:57,929 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:57,929 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:57,929 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:57,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:57,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:57,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:57,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:57,929 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:54:57,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:57,929 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:54:57,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:54:57,929 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:57,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:57,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:57,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:57,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:57,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:57,930 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:57,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:57,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:57,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:57,932 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:57,932 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:54:57,932 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:57,932 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:54:57,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:54:57,932 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:54:57,932 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-05 19:54:57,932 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-05 19:54:57,932 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:54:57,932 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:57,933 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:54:57,933 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:57,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:54:57,933 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:57,933 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-05 19:54:57,933 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:57,933 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-05 19:54:57,933 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:58,033 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-05 19:54:58,034 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-05 19:54:58,037 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-05 19:54:58,039 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-03-05 19:54:58,039 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:58,039 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-05 19:54:58,039 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:54:58,039 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-05 19:54:58,039 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-05 19:54:58,039 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:58,039 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:58,041 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:58,041 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:54:58,041 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:54:58,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:54:58,042 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:58,042 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-05 19:54:58,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:58,042 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:58,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-05 19:54:58,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:58,042 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:58,043 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-03-05 19:54:58,043 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:58,043 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-05 19:54:58,043 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:54:58,043 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-05 19:54:58,043 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-05 19:54:58,043 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-05 19:54:58,043 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678046094042.b9c878f961e0a32027f8960cfe26b4eb. 2023-03-05 19:54:58,043 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678046094042.b9c878f961e0a32027f8960cfe26b4eb. started... 2023-03-05 19:54:58,043 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing b9c878f961e0a32027f8960cfe26b4eb 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-05 19:54:58,457 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205044f36a46f5f4c8fb53b4498f1919849 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151205044f36a46f5f4c8fb53b4498f1919849 2023-03-05 19:54:58,863 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/.tmp/family1/efa6ee9a2a9d4325ba1168737ce9cdd8 2023-03-05 19:54:58,867 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/.tmp/family1/efa6ee9a2a9d4325ba1168737ce9cdd8 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/family1/efa6ee9a2a9d4325ba1168737ce9cdd8 2023-03-05 19:54:58,870 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/family1/efa6ee9a2a9d4325ba1168737ce9cdd8, entries=1, sequenceid=17, filesize=5.0 K 2023-03-05 19:54:58,871 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for b9c878f961e0a32027f8960cfe26b4eb in 828ms, sequenceid=17, compaction requested=true 2023-03-05 19:54:58,871 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for b9c878f961e0a32027f8960cfe26b4eb: 2023-03-05 19:54:58,871 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678046094042.b9c878f961e0a32027f8960cfe26b4eb. 2023-03-05 19:54:58,871 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-05 19:54:58,871 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-05 19:54:58,871 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:58,871 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-05 19:54:58,871 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-05 19:54:58,875 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:58,875 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:58,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:58,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:58,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:58,875 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:58,875 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-05 19:54:58,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:58,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:58,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:58,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:58,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:58,876 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-05 19:54:58,876 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5c3f51dd[Count = 0] remaining members to acquire global barrier 2023-03-05 19:54:58,876 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-05 19:54:58,877 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:58,878 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:58,878 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:58,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:58,878 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-05 19:54:58,878 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-05 19:54:58,878 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,37037,1678046018983' in zk 2023-03-05 19:54:58,878 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:58,878 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-05 19:54:58,880 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-05 19:54:58,880 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:58,880 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:54:58,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:58,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:58,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:58,880 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-05 19:54:58,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:58,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:58,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:58,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:58,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:58,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:58,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:58,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,37037,1678046018983': 2023-03-05 19:54:58,882 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-05 19:54:58,882 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-05 19:54:58,882 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-05 19:54:58,882 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-05 19:54:58,882 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:58,882 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-05 19:54:58,884 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:58,884 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:58,884 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:58,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:58,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:58,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:58,884 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:54:58,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:58,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:58,884 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:58,884 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:54:58,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:54:58,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:58,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:58,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:58,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:58,885 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:58,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:58,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:58,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:58,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:58,887 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:58,887 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:54:58,887 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:58,888 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:54:58,888 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-03-05 19:54:58,888 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:54:58,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:54:58,888 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-05 19:54:58,888 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-05 19:54:58,888 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:58,888 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:54:58,888 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:54:58,888 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:58,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:54:58,888 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:58,888 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:58,888 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-05 19:54:58,888 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-05 19:54:58,888 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:58,988 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-05 19:54:58,989 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-05 19:54:58,992 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-05 19:54:58,994 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-03-05 19:54:58,994 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:58,994 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-05 19:54:58,994 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:54:58,994 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-05 19:54:58,994 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-05 19:54:58,995 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:58,995 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:58,996 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:58,996 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:54:58,996 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:54:58,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:54:58,996 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:58,996 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-05 19:54:58,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:58,997 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:58,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-05 19:54:58,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:58,997 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:58,997 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-03-05 19:54:58,997 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:58,997 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-05 19:54:58,997 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:54:58,997 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-05 19:54:58,997 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-05 19:54:58,997 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-05 19:54:58,998 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678046094042.b9c878f961e0a32027f8960cfe26b4eb. 2023-03-05 19:54:58,998 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678046094042.b9c878f961e0a32027f8960cfe26b4eb. started... 2023-03-05 19:54:58,998 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing b9c878f961e0a32027f8960cfe26b4eb 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-05 19:54:59,409 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151228d3d5a5917ecb459099bf9748b6ab3a91 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151228d3d5a5917ecb459099bf9748b6ab3a91 2023-03-05 19:54:59,814 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/.tmp/family1/280c68abdae644cfac10a599c21d794b 2023-03-05 19:54:59,819 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/.tmp/family1/280c68abdae644cfac10a599c21d794b as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/family1/280c68abdae644cfac10a599c21d794b 2023-03-05 19:54:59,823 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/family1/280c68abdae644cfac10a599c21d794b, entries=1, sequenceid=21, filesize=5.0 K 2023-03-05 19:54:59,823 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for b9c878f961e0a32027f8960cfe26b4eb in 825ms, sequenceid=21, compaction requested=true 2023-03-05 19:54:59,823 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for b9c878f961e0a32027f8960cfe26b4eb: 2023-03-05 19:54:59,823 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678046094042.b9c878f961e0a32027f8960cfe26b4eb. 2023-03-05 19:54:59,823 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-05 19:54:59,824 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-05 19:54:59,824 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:59,824 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-05 19:54:59,824 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-05 19:54:59,826 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:59,826 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:59,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:59,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:59,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:59,827 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:59,827 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-05 19:54:59,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:59,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:59,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:59,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:59,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:59,828 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-05 19:54:59,828 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2004bffa[Count = 0] remaining members to acquire global barrier 2023-03-05 19:54:59,828 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-05 19:54:59,828 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:59,829 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:59,829 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:59,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:59,829 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-05 19:54:59,829 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-05 19:54:59,829 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,37037,1678046018983' in zk 2023-03-05 19:54:59,830 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:59,830 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-05 19:54:59,831 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:59,831 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-05 19:54:59,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:59,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:59,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:59,831 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:54:59,831 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-05 19:54:59,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:59,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:59,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:59,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:59,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:59,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:59,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:59,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,37037,1678046018983': 2023-03-05 19:54:59,833 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-05 19:54:59,833 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-05 19:54:59,833 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-05 19:54:59,833 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-05 19:54:59,833 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:59,833 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-05 19:54:59,834 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:59,834 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:59,834 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:54:59,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:59,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:54:59,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:54:59,834 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:59,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:59,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:54:59,835 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:54:59,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:54:59,835 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:59,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:59,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:59,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:54:59,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:59,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:59,836 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:59,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:54:59,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:59,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:59,839 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:59,839 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:59,839 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:59,839 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:59,839 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:59,839 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:59,839 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:59,839 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:54:59,839 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:54:59,839 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-03-05 19:54:59,839 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-05 19:54:59,840 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-05 19:54:59,839 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:54:59,839 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:54:59,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:54:59,840 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-05 19:54:59,840 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-05 19:54:59,840 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:54:59,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:54:59,940 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-05 19:54:59,941 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-05 19:54:59,944 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-05 19:54:59,946 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-03-05 19:54:59,946 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:59,946 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-05 19:54:59,946 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:54:59,946 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-05 19:54:59,946 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-05 19:54:59,946 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:59,946 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:59,947 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:59,947 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:54:59,948 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:54:59,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:54:59,948 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:54:59,948 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-05 19:54:59,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:59,948 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:59,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-05 19:54:59,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:59,948 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:59,949 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-03-05 19:54:59,949 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:54:59,949 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-05 19:54:59,949 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:54:59,949 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-05 19:54:59,949 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-05 19:54:59,949 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-05 19:54:59,949 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678046094042.b9c878f961e0a32027f8960cfe26b4eb. 2023-03-05 19:54:59,949 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678046094042.b9c878f961e0a32027f8960cfe26b4eb. started... 2023-03-05 19:54:59,949 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing b9c878f961e0a32027f8960cfe26b4eb 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-05 19:55:00,362 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231fc6113d33f644290970f694a6c16c7f7 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151231fc6113d33f644290970f694a6c16c7f7 2023-03-05 19:55:00,767 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/.tmp/family1/afd836895ad74c61b7049114fd40b4c7 2023-03-05 19:55:00,771 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/.tmp/family1/afd836895ad74c61b7049114fd40b4c7 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/family1/afd836895ad74c61b7049114fd40b4c7 2023-03-05 19:55:00,775 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/family1/afd836895ad74c61b7049114fd40b4c7, entries=1, sequenceid=25, filesize=5.0 K 2023-03-05 19:55:00,776 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for b9c878f961e0a32027f8960cfe26b4eb in 827ms, sequenceid=25, compaction requested=true 2023-03-05 19:55:00,776 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for b9c878f961e0a32027f8960cfe26b4eb: 2023-03-05 19:55:00,776 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678046094042.b9c878f961e0a32027f8960cfe26b4eb. 2023-03-05 19:55:00,776 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-05 19:55:00,776 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-05 19:55:00,776 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:00,776 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-05 19:55:00,776 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-05 19:55:00,778 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:00,778 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:00,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:00,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:55:00,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:55:00,778 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:00,778 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-05 19:55:00,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:55:00,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:55:00,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:00,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:00,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:55:00,779 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-05 19:55:00,779 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@37d02095[Count = 0] remaining members to acquire global barrier 2023-03-05 19:55:00,779 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-05 19:55:00,779 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:00,780 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:00,780 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:00,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:00,781 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-05 19:55:00,781 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-05 19:55:00,781 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:00,781 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-05 19:55:00,781 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,37037,1678046018983' in zk 2023-03-05 19:55:00,783 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:00,783 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-05 19:55:00,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:00,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:55:00,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:55:00,783 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:55:00,784 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-05 19:55:00,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:55:00,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:55:00,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:00,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:00,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:55:00,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:00,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:00,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,37037,1678046018983': 2023-03-05 19:55:00,786 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-05 19:55:00,786 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-05 19:55:00,786 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-05 19:55:00,786 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-05 19:55:00,786 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:00,786 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-05 19:55:00,787 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:00,787 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:00,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:00,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:55:00,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:55:00,787 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:00,787 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:55:00,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:00,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:55:00,788 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:00,788 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:55:00,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:55:00,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:00,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:00,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:55:00,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:00,789 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:00,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:00,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:55:00,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:00,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:00,793 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:55:00,793 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:55:00,793 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:55:00,793 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:00,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:55:00,793 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:00,794 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:00,794 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:00,794 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:55:00,794 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-05 19:55:00,794 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-05 19:55:00,794 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:00,794 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:55:00,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:55:00,794 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:00,794 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:00,794 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-05 19:55:00,794 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-05 19:55:00,894 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-05 19:55:00,895 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-05 19:55:00,899 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-05 19:55:00,900 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-03-05 19:55:00,900 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:00,900 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-05 19:55:00,901 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:55:00,901 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-05 19:55:00,901 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-05 19:55:00,901 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:00,901 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:00,906 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:55:00,906 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:00,906 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:55:00,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:55:00,906 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:00,906 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-05 19:55:00,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:00,907 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:00,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-05 19:55:00,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:00,907 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:00,907 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-03-05 19:55:00,907 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:00,907 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-05 19:55:00,907 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:55:00,907 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-05 19:55:00,908 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-05 19:55:00,908 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-05 19:55:00,908 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678046094042.b9c878f961e0a32027f8960cfe26b4eb. 2023-03-05 19:55:00,908 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678046094042.b9c878f961e0a32027f8960cfe26b4eb. started... 2023-03-05 19:55:00,908 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing b9c878f961e0a32027f8960cfe26b4eb 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-05 19:55:01,319 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160101329f1ad45a584bc7b8b47f9840bf0dd4 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20160101329f1ad45a584bc7b8b47f9840bf0dd4 2023-03-05 19:55:01,724 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/.tmp/family1/887b52f61b4245a59d323f7aa0792f18 2023-03-05 19:55:01,728 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/.tmp/family1/887b52f61b4245a59d323f7aa0792f18 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/family1/887b52f61b4245a59d323f7aa0792f18 2023-03-05 19:55:01,731 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/family1/887b52f61b4245a59d323f7aa0792f18, entries=1, sequenceid=29, filesize=5.0 K 2023-03-05 19:55:01,732 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for b9c878f961e0a32027f8960cfe26b4eb in 824ms, sequenceid=29, compaction requested=true 2023-03-05 19:55:01,732 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for b9c878f961e0a32027f8960cfe26b4eb: 2023-03-05 19:55:01,732 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678046094042.b9c878f961e0a32027f8960cfe26b4eb. 2023-03-05 19:55:01,732 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-05 19:55:01,732 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-05 19:55:01,732 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:01,733 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-05 19:55:01,733 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-05 19:55:01,734 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:01,734 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:01,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:01,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:55:01,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:55:01,734 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:01,734 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-05 19:55:01,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:55:01,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:55:01,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:01,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:01,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:55:01,735 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-05 19:55:01,735 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5801a046[Count = 0] remaining members to acquire global barrier 2023-03-05 19:55:01,735 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-05 19:55:01,736 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:01,737 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:01,737 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:01,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:01,737 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-05 19:55:01,737 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-05 19:55:01,737 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,37037,1678046018983' in zk 2023-03-05 19:55:01,737 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:01,737 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-05 19:55:01,738 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-05 19:55:01,738 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:01,738 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:55:01,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:01,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:55:01,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:55:01,738 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-05 19:55:01,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:55:01,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:55:01,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:01,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:01,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:55:01,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:01,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:01,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,37037,1678046018983': 2023-03-05 19:55:01,740 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-05 19:55:01,740 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-05 19:55:01,740 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-05 19:55:01,740 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-05 19:55:01,740 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:01,740 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-05 19:55:01,742 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:01,742 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:01,742 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:01,742 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:55:01,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:01,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:01,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:55:01,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:55:01,743 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:55:01,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:55:01,743 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:01,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:55:01,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:01,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:01,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:55:01,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:01,744 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:01,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:01,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:55:01,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:01,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:01,746 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:01,746 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:55:01,746 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:01,746 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:55:01,746 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:55:01,746 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-03-05 19:55:01,746 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:01,746 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:55:01,746 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:01,746 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-05 19:55:01,746 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-05 19:55:01,746 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:01,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:55:01,746 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:01,746 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:01,746 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-05 19:55:01,746 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:55:01,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:55:01,747 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-05 19:55:01,847 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-05 19:55:01,847 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-05 19:55:01,851 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-05 19:55:01,853 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-03-05 19:55:01,853 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:01,853 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-05 19:55:01,853 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:55:01,853 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-05 19:55:01,853 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-05 19:55:01,854 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:01,854 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:01,855 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:01,855 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:55:01,855 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:55:01,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:55:01,855 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:01,855 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-05 19:55:01,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:01,856 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:01,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-05 19:55:01,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:01,856 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:01,857 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-03-05 19:55:01,857 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:01,857 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-05 19:55:01,857 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:55:01,857 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-05 19:55:01,857 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-05 19:55:01,857 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-05 19:55:01,857 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678046094042.b9c878f961e0a32027f8960cfe26b4eb. 2023-03-05 19:55:01,857 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678046094042.b9c878f961e0a32027f8960cfe26b4eb. started... 2023-03-05 19:55:01,857 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing b9c878f961e0a32027f8960cfe26b4eb 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-05 19:55:02,273 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2016010374cd7245f70b4d979d0378929f65adbc to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2016010374cd7245f70b4d979d0378929f65adbc 2023-03-05 19:55:02,678 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/.tmp/family1/d4d54566f12c4b5c9ba8e920cdc25cfc 2023-03-05 19:55:02,683 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/.tmp/family1/d4d54566f12c4b5c9ba8e920cdc25cfc as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/family1/d4d54566f12c4b5c9ba8e920cdc25cfc 2023-03-05 19:55:02,686 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/family1/d4d54566f12c4b5c9ba8e920cdc25cfc, entries=1, sequenceid=33, filesize=5.0 K 2023-03-05 19:55:02,687 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for b9c878f961e0a32027f8960cfe26b4eb in 830ms, sequenceid=33, compaction requested=true 2023-03-05 19:55:02,687 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for b9c878f961e0a32027f8960cfe26b4eb: 2023-03-05 19:55:02,687 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678046094042.b9c878f961e0a32027f8960cfe26b4eb. 2023-03-05 19:55:02,687 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-05 19:55:02,687 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-05 19:55:02,687 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:02,687 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-05 19:55:02,687 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-05 19:55:02,689 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:02,689 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:02,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:02,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:55:02,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:55:02,689 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:02,689 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-05 19:55:02,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:55:02,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:55:02,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:02,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:02,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:55:02,690 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-05 19:55:02,690 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7e16b322[Count = 0] remaining members to acquire global barrier 2023-03-05 19:55:02,690 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-05 19:55:02,690 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:02,692 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:02,692 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:02,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:02,693 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-05 19:55:02,693 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-05 19:55:02,693 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,37037,1678046018983' in zk 2023-03-05 19:55:02,693 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:02,693 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-05 19:55:02,694 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-05 19:55:02,694 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:02,694 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:55:02,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:02,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:55:02,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:55:02,694 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-05 19:55:02,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:55:02,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:55:02,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:02,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:02,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:55:02,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:02,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:02,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,37037,1678046018983': 2023-03-05 19:55:02,696 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-05 19:55:02,696 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-05 19:55:02,696 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-05 19:55:02,696 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-05 19:55:02,696 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:02,696 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-05 19:55:02,698 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:02,698 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:02,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:02,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:55:02,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:55:02,698 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:02,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:02,698 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:55:02,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:55:02,698 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:02,698 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:55:02,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:55:02,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:02,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:02,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:55:02,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:02,699 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:02,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:02,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:55:02,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:02,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:02,701 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:02,701 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:55:02,701 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:02,701 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:55:02,701 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:02,701 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:55:02,701 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:55:02,701 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-05 19:55:02,702 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-05 19:55:02,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:55:02,701 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:02,702 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:02,702 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:02,702 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:02,702 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-05 19:55:02,702 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-05 19:55:02,702 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:55:02,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:55:02,802 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-05 19:55:02,803 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-05 19:55:02,806 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-05 19:55:02,808 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-03-05 19:55:02,808 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:02,808 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-05 19:55:02,808 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:55:02,808 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-05 19:55:02,808 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-05 19:55:02,809 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:02,809 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:02,811 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:55:02,811 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:02,811 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:55:02,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:55:02,811 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:02,811 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-05 19:55:02,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:02,812 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:02,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-05 19:55:02,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:02,812 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:02,812 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-03-05 19:55:02,812 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:02,812 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-05 19:55:02,812 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:55:02,812 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-05 19:55:02,813 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-05 19:55:02,813 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-05 19:55:02,813 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678046094042.b9c878f961e0a32027f8960cfe26b4eb. 2023-03-05 19:55:02,813 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678046094042.b9c878f961e0a32027f8960cfe26b4eb. started... 2023-03-05 19:55:02,813 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing b9c878f961e0a32027f8960cfe26b4eb 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-05 19:55:03,224 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20150907cdd31ce25efc46f2a38c632770e6bcd7 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20150907cdd31ce25efc46f2a38c632770e6bcd7 2023-03-05 19:55:03,629 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/.tmp/family1/96524087c0674f1db9061d37a674f78f 2023-03-05 19:55:03,633 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/.tmp/family1/96524087c0674f1db9061d37a674f78f as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/family1/96524087c0674f1db9061d37a674f78f 2023-03-05 19:55:03,636 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/family1/96524087c0674f1db9061d37a674f78f, entries=1, sequenceid=37, filesize=5.0 K 2023-03-05 19:55:03,637 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for b9c878f961e0a32027f8960cfe26b4eb in 824ms, sequenceid=37, compaction requested=true 2023-03-05 19:55:03,637 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for b9c878f961e0a32027f8960cfe26b4eb: 2023-03-05 19:55:03,637 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678046094042.b9c878f961e0a32027f8960cfe26b4eb. 2023-03-05 19:55:03,637 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-05 19:55:03,637 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-05 19:55:03,637 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:03,637 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-05 19:55:03,637 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-05 19:55:03,639 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:03,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:03,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:55:03,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:55:03,639 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:03,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:55:03,639 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:03,639 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-05 19:55:03,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:55:03,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:03,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:03,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:55:03,641 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-05 19:55:03,641 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7be82875[Count = 0] remaining members to acquire global barrier 2023-03-05 19:55:03,641 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-05 19:55:03,641 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:03,642 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:03,642 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:03,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:03,642 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-05 19:55:03,642 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-05 19:55:03,642 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,37037,1678046018983' in zk 2023-03-05 19:55:03,642 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:03,642 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-05 19:55:03,643 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-05 19:55:03,643 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:03,643 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:55:03,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:03,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:55:03,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:55:03,643 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-05 19:55:03,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:55:03,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:55:03,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:03,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:03,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:55:03,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:03,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:03,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,37037,1678046018983': 2023-03-05 19:55:03,645 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-05 19:55:03,645 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-05 19:55:03,646 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-05 19:55:03,646 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-05 19:55:03,646 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:03,646 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-05 19:55:03,648 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:03,648 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:03,648 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:55:03,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:03,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:55:03,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:55:03,648 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:03,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:03,649 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:55:03,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:55:03,649 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:03,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:55:03,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:03,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:03,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:55:03,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:03,650 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:03,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:03,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:55:03,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:03,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:03,652 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:03,652 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:55:03,652 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:03,652 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:03,652 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:03,652 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:03,652 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:55:03,652 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:55:03,652 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:55:03,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:55:03,652 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-05 19:55:03,652 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-05 19:55:03,652 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:03,652 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:03,652 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-03-05 19:55:03,652 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:55:03,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:55:03,652 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-05 19:55:03,653 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-05 19:55:03,753 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-05 19:55:03,754 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-05 19:55:03,757 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-05 19:55:03,759 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-03-05 19:55:03,759 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:03,759 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-05 19:55:03,759 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:55:03,759 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-05 19:55:03,759 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-05 19:55:03,759 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:03,759 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:03,760 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:55:03,761 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:03,761 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:55:03,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:55:03,761 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:03,761 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-05 19:55:03,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:03,761 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:03,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-05 19:55:03,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:03,762 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:03,762 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-03-05 19:55:03,762 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:03,762 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-05 19:55:03,762 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:55:03,762 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-05 19:55:03,762 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-05 19:55:03,762 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-05 19:55:03,762 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678046094042.b9c878f961e0a32027f8960cfe26b4eb. 2023-03-05 19:55:03,762 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678046094042.b9c878f961e0a32027f8960cfe26b4eb. started... 2023-03-05 19:55:03,762 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing b9c878f961e0a32027f8960cfe26b4eb 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-05 19:55:04,175 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151120138a53602a1249ef9362f50f52690232 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151120138a53602a1249ef9362f50f52690232 2023-03-05 19:55:04,580 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/.tmp/family1/952cfd007a93494e89fce8497eb745f6 2023-03-05 19:55:04,584 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/.tmp/family1/952cfd007a93494e89fce8497eb745f6 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/family1/952cfd007a93494e89fce8497eb745f6 2023-03-05 19:55:04,589 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/family1/952cfd007a93494e89fce8497eb745f6, entries=1, sequenceid=41, filesize=5.0 K 2023-03-05 19:55:04,589 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for b9c878f961e0a32027f8960cfe26b4eb in 827ms, sequenceid=41, compaction requested=true 2023-03-05 19:55:04,589 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for b9c878f961e0a32027f8960cfe26b4eb: 2023-03-05 19:55:04,589 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678046094042.b9c878f961e0a32027f8960cfe26b4eb. 2023-03-05 19:55:04,590 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-05 19:55:04,590 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-05 19:55:04,590 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:04,590 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-05 19:55:04,590 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-05 19:55:04,591 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:04,591 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:04,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:04,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:55:04,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:55:04,592 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:04,592 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-05 19:55:04,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:55:04,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:55:04,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:04,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:04,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:55:04,593 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-05 19:55:04,593 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4d0a7b15[Count = 0] remaining members to acquire global barrier 2023-03-05 19:55:04,593 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-05 19:55:04,593 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:04,595 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:04,595 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:04,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:04,595 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-05 19:55:04,596 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-05 19:55:04,596 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,37037,1678046018983' in zk 2023-03-05 19:55:04,596 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:04,596 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-05 19:55:04,597 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-05 19:55:04,597 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:04,597 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:55:04,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:04,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:55:04,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:55:04,597 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-05 19:55:04,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:55:04,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:55:04,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:04,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:04,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:55:04,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:04,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:04,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,37037,1678046018983': 2023-03-05 19:55:04,599 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-05 19:55:04,599 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-05 19:55:04,599 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-05 19:55:04,599 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-05 19:55:04,599 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:04,599 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-05 19:55:04,600 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:04,600 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:04,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:04,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:55:04,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:55:04,601 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:04,601 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:55:04,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:04,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:55:04,601 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:04,601 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:55:04,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:55:04,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:04,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:04,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:55:04,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:04,602 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:04,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:04,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:55:04,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:04,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:04,604 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:04,604 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:55:04,604 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:04,604 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:55:04,604 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:55:04,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:55:04,604 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-05 19:55:04,604 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-05 19:55:04,604 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:04,604 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:04,604 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:04,604 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-03-05 19:55:04,604 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:55:04,604 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:04,604 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:55:04,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:55:04,604 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:04,604 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-05 19:55:04,604 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-05 19:55:04,705 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-05 19:55:04,705 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-05 19:55:04,707 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=38221] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:04,708 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] procedure2.ProcedureExecutor(1029): Stored pid=58, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-03-05 19:55:04,709 DEBUG [PEWorker-4] locking.LockProcedure(309): LOCKED pid=58, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-03-05 19:55:04,709 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-05 19:55:04,709 INFO [PEWorker-4] procedure2.TimeoutExecutorThread(81): ADDED pid=58, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678046704709 2023-03-05 19:55:04,712 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-05 19:55:04,712 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column 'family1' 2023-03-05 19:55:04,712 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-05 19:55:04,713 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] zookeeper.ReadOnlyZKClient(139): Connect 0x6872a4ff to 127.0.0.1:62957 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-05 19:55:04,718 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6a4ecc14, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-05 19:55:04,718 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-05 19:55:05,551 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2016010335a83e5997d7436b82a69c63db101692 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2016010335a83e5997d7436b82a69c63db101692 2023-03-05 19:55:05,552 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-05 19:55:05,565 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-05 19:55:05,566 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:58960, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-05 19:55:05,573 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-05 19:55:05,574 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:43754, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-05 19:55:05,582 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228/family1/d7a454b9f6194a48aa3afea77f167d2c first=Optional[r3] last=Optional[r6] 2023-03-05 19:55:05,583 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678046094042.b9c878f961e0a32027f8960cfe26b4eb., hostname=jenkins-hbase4.apache.org,37037,1678046018983, seqNum=2 for row with hfile group [{family1,hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228/family1/d7a454b9f6194a48aa3afea77f167d2c}] 2023-03-05 19:55:05,586 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(763): Validating hfile at hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228/family1/d7a454b9f6194a48aa3afea77f167d2c for inclusion in b9c878f961e0a32027f8960cfe26b4eb/family1 2023-03-05 19:55:05,588 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-03-05 19:55:05,589 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(777): Region bounds: first= last= 2023-03-05 19:55:05,589 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HRegion(2446): Flush status journal for b9c878f961e0a32027f8960cfe26b4eb: 2023-03-05 19:55:05,589 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228/family1/d7a454b9f6194a48aa3afea77f167d2c to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__f4lbrr27af9imr6ocvnjpelbj7rva40irdupphktgio3i11gkqvfpqoflr1k55t6/family1/d7a454b9f6194a48aa3afea77f167d2c 2023-03-05 19:55:05,591 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__f4lbrr27af9imr6ocvnjpelbj7rva40irdupphktgio3i11gkqvfpqoflr1k55t6/family1/d7a454b9f6194a48aa3afea77f167d2c as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/family1/3cfad6337a694ac6bf5de0ded9922028_SeqId_44_ 2023-03-05 19:55:05,592 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(849): Loaded HFile hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__f4lbrr27af9imr6ocvnjpelbj7rva40irdupphktgio3i11gkqvfpqoflr1k55t6/family1/d7a454b9f6194a48aa3afea77f167d2c into b9c878f961e0a32027f8960cfe26b4eb/family1 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/family1/3cfad6337a694ac6bf5de0ded9922028_SeqId_44_ - updating store file list. 2023-03-05 19:55:05,595 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(882): Loaded HFile hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/family1/3cfad6337a694ac6bf5de0ded9922028_SeqId_44_ into b9c878f961e0a32027f8960cfe26b4eb/family1 2023-03-05 19:55:05,595 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(855): Successfully loaded hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__f4lbrr27af9imr6ocvnjpelbj7rva40irdupphktgio3i11gkqvfpqoflr1k55t6/family1/d7a454b9f6194a48aa3afea77f167d2c into b9c878f961e0a32027f8960cfe26b4eb/family1 (new location: hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/family1/3cfad6337a694ac6bf5de0ded9922028_SeqId_44_) 2023-03-05 19:55:05,595 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__f4lbrr27af9imr6ocvnjpelbj7rva40irdupphktgio3i11gkqvfpqoflr1k55t6/family1/d7a454b9f6194a48aa3afea77f167d2c 2023-03-05 19:55:05,595 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-03-05 19:55:05,598 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51552 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-03-05 19:55:05,598 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(1912): b9c878f961e0a32027f8960cfe26b4eb/family1 is initiating minor compaction 2023-03-05 19:55:05,598 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@2ab6a7bf; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-05 19:55:05,598 INFO [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of b9c878f961e0a32027f8960cfe26b4eb/family1 in testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678046094042.b9c878f961e0a32027f8960cfe26b4eb. 2023-03-05 19:55:05,598 INFO [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/family1/77334e1802b241ab971852cbeb33bac5, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/family1/0701549599d349ffada783dd7788b746, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/family1/efa6ee9a2a9d4325ba1168737ce9cdd8, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/family1/280c68abdae644cfac10a599c21d794b, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/family1/afd836895ad74c61b7049114fd40b4c7, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/family1/887b52f61b4245a59d323f7aa0792f18, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/family1/d4d54566f12c4b5c9ba8e920cdc25cfc, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/family1/96524087c0674f1db9061d37a674f78f, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/family1/952cfd007a93494e89fce8497eb745f6, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/family1/3cfad6337a694ac6bf5de0ded9922028_SeqId_44_] into tmpdir=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/.tmp, totalSize=50.3 K 2023-03-05 19:55:05,599 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.Compactor(207): Compacting 77334e1802b241ab971852cbeb33bac5, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-03-05 19:55:05,599 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.Compactor(207): Compacting 0701549599d349ffada783dd7788b746, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=13 2023-03-05 19:55:05,599 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.Compactor(207): Compacting efa6ee9a2a9d4325ba1168737ce9cdd8, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=17 2023-03-05 19:55:05,599 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.Compactor(207): Compacting 280c68abdae644cfac10a599c21d794b, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=21 2023-03-05 19:55:05,599 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.Compactor(207): Compacting afd836895ad74c61b7049114fd40b4c7, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=25 2023-03-05 19:55:05,599 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.Compactor(207): Compacting 887b52f61b4245a59d323f7aa0792f18, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=29 2023-03-05 19:55:05,599 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.Compactor(207): Compacting d4d54566f12c4b5c9ba8e920cdc25cfc, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33 2023-03-05 19:55:05,599 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.Compactor(207): Compacting 96524087c0674f1db9061d37a674f78f, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=37 2023-03-05 19:55:05,599 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.Compactor(207): Compacting 952cfd007a93494e89fce8497eb745f6, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=41 2023-03-05 19:55:05,599 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.Compactor(207): Compacting 3cfad6337a694ac6bf5de0ded9922028_SeqId_44_, keycount=4, bloomtype=ROW, size=5.6 K, encoding=NONE, compression=NONE, seqNum=45 2023-03-05 19:55:05,600 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-05 19:55:05,601 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-05 19:55:05,603 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151228d3d5a5917ecb459099bf9748b6ab3a91 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151228d3d5a5917ecb459099bf9748b6ab3a91 2023-03-05 19:55:05,604 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151231fc6113d33f644290970f694a6c16c7f7 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151231fc6113d33f644290970f694a6c16c7f7 2023-03-05 19:55:05,605 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20160101329f1ad45a584bc7b8b47f9840bf0dd4 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20160101329f1ad45a584bc7b8b47f9840bf0dd4 2023-03-05 19:55:05,607 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2016010374cd7245f70b4d979d0378929f65adbc to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2016010374cd7245f70b4d979d0378929f65adbc 2023-03-05 19:55:05,607 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 4 expired mob files are deleted 2023-03-05 19:55:05,607 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-03-05 19:55:05,607 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-05 19:55:05,607 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-05 19:55:05,608 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-05 19:55:05,608 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-05 19:55:05,608 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-05 19:55:05,608 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-05 19:55:05,608 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-05 19:55:05,608 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-05 19:55:05,644 INFO [RS:0;jenkins-hbase4:37037-shortCompactions-0] throttle.PressureAwareThroughputController(145): b9c878f961e0a32027f8960cfe26b4eb#family1#compaction#72 average throughput is 0.79 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-05 19:55:06,062 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/.tmp/family1/23e83c6f75fa4c35bc4bb602947f83c9 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/family1/23e83c6f75fa4c35bc4bb602947f83c9 2023-03-05 19:55:06,067 INFO [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in b9c878f961e0a32027f8960cfe26b4eb/family1 of b9c878f961e0a32027f8960cfe26b4eb into 23e83c6f75fa4c35bc4bb602947f83c9(size=6.8 K), total size for store is 197.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-05 19:55:06,067 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for b9c878f961e0a32027f8960cfe26b4eb: 2023-03-05 19:55:06,067 INFO [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678046094042.b9c878f961e0a32027f8960cfe26b4eb., storeName=family1, fileCount=10, fileSize=50.3 K (5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.6 K), priority=2, time=1678046105598; duration=0sec 2023-03-05 19:55:06,067 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-05 19:55:06,109 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/family1/77334e1802b241ab971852cbeb33bac5, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/family1/0701549599d349ffada783dd7788b746, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/family1/efa6ee9a2a9d4325ba1168737ce9cdd8, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/family1/280c68abdae644cfac10a599c21d794b, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/family1/afd836895ad74c61b7049114fd40b4c7, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/family1/887b52f61b4245a59d323f7aa0792f18, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/family1/d4d54566f12c4b5c9ba8e920cdc25cfc, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/family1/96524087c0674f1db9061d37a674f78f, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/family1/952cfd007a93494e89fce8497eb745f6, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/family1/3cfad6337a694ac6bf5de0ded9922028_SeqId_44_] to archive 2023-03-05 19:55:06,110 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-05 19:55:06,111 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/family1/77334e1802b241ab971852cbeb33bac5 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/family1/77334e1802b241ab971852cbeb33bac5 2023-03-05 19:55:06,112 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/family1/0701549599d349ffada783dd7788b746 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/family1/0701549599d349ffada783dd7788b746 2023-03-05 19:55:06,113 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/family1/efa6ee9a2a9d4325ba1168737ce9cdd8 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/family1/efa6ee9a2a9d4325ba1168737ce9cdd8 2023-03-05 19:55:06,114 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/family1/280c68abdae644cfac10a599c21d794b to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/family1/280c68abdae644cfac10a599c21d794b 2023-03-05 19:55:06,115 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/family1/afd836895ad74c61b7049114fd40b4c7 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/family1/afd836895ad74c61b7049114fd40b4c7 2023-03-05 19:55:06,115 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/family1/887b52f61b4245a59d323f7aa0792f18 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/family1/887b52f61b4245a59d323f7aa0792f18 2023-03-05 19:55:06,116 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/family1/d4d54566f12c4b5c9ba8e920cdc25cfc to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/family1/d4d54566f12c4b5c9ba8e920cdc25cfc 2023-03-05 19:55:06,117 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/family1/96524087c0674f1db9061d37a674f78f to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/family1/96524087c0674f1db9061d37a674f78f 2023-03-05 19:55:06,118 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/family1/952cfd007a93494e89fce8497eb745f6 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/family1/952cfd007a93494e89fce8497eb745f6 2023-03-05 19:55:06,119 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/family1/3cfad6337a694ac6bf5de0ded9922028_SeqId_44_ to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/family1/3cfad6337a694ac6bf5de0ded9922028_SeqId_44_ 2023-03-05 19:55:06,464 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512050f54da8e79b14bf594e37da6b518878b to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201512050f54da8e79b14bf594e37da6b518878b 2023-03-05 19:55:06,465 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-05 19:55:06,489 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130/family1/0c4b64a20ccf46faac16cb5edc8c5b80 first=Optional[r00] last=Optional[r2] 2023-03-05 19:55:06,489 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678046094042.b9c878f961e0a32027f8960cfe26b4eb., hostname=jenkins-hbase4.apache.org,37037,1678046018983, seqNum=2 for row with hfile group [{family1,hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130/family1/0c4b64a20ccf46faac16cb5edc8c5b80}] 2023-03-05 19:55:06,491 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(763): Validating hfile at hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130/family1/0c4b64a20ccf46faac16cb5edc8c5b80 for inclusion in b9c878f961e0a32027f8960cfe26b4eb/family1 2023-03-05 19:55:06,496 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(775): HFile bounds: first=r00 last=r2 2023-03-05 19:55:06,496 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(777): Region bounds: first= last= 2023-03-05 19:55:06,496 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HRegion(2446): Flush status journal for b9c878f961e0a32027f8960cfe26b4eb: 2023-03-05 19:55:06,497 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130/family1/0c4b64a20ccf46faac16cb5edc8c5b80 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__iqrop7mou01fk3mf9lqs45nerf77jcjcsvtial5jqc857ll6mleq4mpm4j89smck/family1/0c4b64a20ccf46faac16cb5edc8c5b80 2023-03-05 19:55:06,499 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__iqrop7mou01fk3mf9lqs45nerf77jcjcsvtial5jqc857ll6mleq4mpm4j89smck/family1/0c4b64a20ccf46faac16cb5edc8c5b80 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/family1/0fb0170e1b85486d91db4848320a4adc_SeqId_47_ 2023-03-05 19:55:06,500 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(849): Loaded HFile hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__iqrop7mou01fk3mf9lqs45nerf77jcjcsvtial5jqc857ll6mleq4mpm4j89smck/family1/0c4b64a20ccf46faac16cb5edc8c5b80 into b9c878f961e0a32027f8960cfe26b4eb/family1 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/family1/0fb0170e1b85486d91db4848320a4adc_SeqId_47_ - updating store file list. 2023-03-05 19:55:06,504 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(882): Loaded HFile hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/family1/0fb0170e1b85486d91db4848320a4adc_SeqId_47_ into b9c878f961e0a32027f8960cfe26b4eb/family1 2023-03-05 19:55:06,504 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(855): Successfully loaded hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__iqrop7mou01fk3mf9lqs45nerf77jcjcsvtial5jqc857ll6mleq4mpm4j89smck/family1/0c4b64a20ccf46faac16cb5edc8c5b80 into b9c878f961e0a32027f8960cfe26b4eb/family1 (new location: hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/family1/0fb0170e1b85486d91db4848320a4adc_SeqId_47_) 2023-03-05 19:55:06,505 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__iqrop7mou01fk3mf9lqs45nerf77jcjcsvtial5jqc857ll6mleq4mpm4j89smck/family1/0c4b64a20ccf46faac16cb5edc8c5b80 2023-03-05 19:55:06,505 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-03-05 19:55:06,506 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 397422 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-03-05 19:55:06,506 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(1912): b9c878f961e0a32027f8960cfe26b4eb/family1 is initiating minor compaction (all files) 2023-03-05 19:55:06,506 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@b91833e; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-05 19:55:06,506 INFO [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of b9c878f961e0a32027f8960cfe26b4eb/family1 in testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678046094042.b9c878f961e0a32027f8960cfe26b4eb. 2023-03-05 19:55:06,506 INFO [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/family1/a1cf7b892fdf49569eaa81f22bfaf698, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/family1/23e83c6f75fa4c35bc4bb602947f83c9, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/family1/0fb0170e1b85486d91db4848320a4adc_SeqId_47_] into tmpdir=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/.tmp, totalSize=388.1 K 2023-03-05 19:55:06,506 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.Compactor(207): Compacting a1cf7b892fdf49569eaa81f22bfaf698, keycount=1000, bloomtype=ROW, size=190.5 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878843274 2023-03-05 19:55:06,507 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.Compactor(207): Compacting 23e83c6f75fa4c35bc4bb602947f83c9, keycount=9, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621243274 2023-03-05 19:55:06,507 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.Compactor(207): Compacting 0fb0170e1b85486d91db4848320a4adc_SeqId_47_, keycount=1002, bloomtype=ROW, size=190.9 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448878843274 2023-03-05 19:55:06,507 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-05 19:55:06,508 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-05 19:55:06,509 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151130f50f2d703f324c25aa44937e76337f51 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151130f50f2d703f324c25aa44937e76337f51 2023-03-05 19:55:06,511 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151201cf1b200636fe4a06b6f23e8335b543a6 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151201cf1b200636fe4a06b6f23e8335b543a6 2023-03-05 19:55:06,512 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151205044f36a46f5f4c8fb53b4498f1919849 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151205044f36a46f5f4c8fb53b4498f1919849 2023-03-05 19:55:06,512 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 3 expired mob files are deleted 2023-03-05 19:55:06,512 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-03-05 19:55:06,513 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-05 19:55:06,513 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-05 19:55:06,513 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-05 19:55:06,514 DEBUG [PEWorker-1] locking.LockProcedure(242): UNLOCKED pid=58, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-03-05 19:55:06,516 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=58, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE in 1.8070 sec 2023-03-05 19:55:06,522 INFO [RS:0;jenkins-hbase4:37037-shortCompactions-0] throttle.PressureAwareThroughputController(145): b9c878f961e0a32027f8960cfe26b4eb#family1#compaction#73 average throughput is 29.94 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-05 19:55:06,582 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.HMaster$12(2454): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily from 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-03-05 19:55:06,583 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure2.ProcedureExecutor(1029): Stored pid=59, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:06,992 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000002 2023-03-05 19:55:06,993 DEBUG [PEWorker-2] util.FSTableDescriptors(519): Deleted hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000001 2023-03-05 19:55:06,993 INFO [PEWorker-2] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000002 2023-03-05 19:55:06,994 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=60, ppid=59, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily}] 2023-03-05 19:55:06,997 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=61, ppid=60, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=b9c878f961e0a32027f8960cfe26b4eb, REOPEN/MOVE}] 2023-03-05 19:55:06,998 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=61, ppid=60, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=b9c878f961e0a32027f8960cfe26b4eb, REOPEN/MOVE 2023-03-05 19:55:06,998 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=61 updating hbase:meta row=b9c878f961e0a32027f8960cfe26b4eb, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:06,998 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678046094042.b9c878f961e0a32027f8960cfe26b4eb.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1678046106998"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678046106998"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678046106998"}]},"ts":"1678046106998"} 2023-03-05 19:55:07,000 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=62, ppid=61, state=RUNNABLE; CloseRegionProcedure b9c878f961e0a32027f8960cfe26b4eb, server=jenkins-hbase4.apache.org,37037,1678046018983}] 2023-03-05 19:55:07,151 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close b9c878f961e0a32027f8960cfe26b4eb 2023-03-05 19:55:07,151 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing b9c878f961e0a32027f8960cfe26b4eb, disabling compactions & flushes 2023-03-05 19:55:07,151 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1858): waiting for 1 compactions to complete for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678046094042.b9c878f961e0a32027f8960cfe26b4eb. 2023-03-05 19:55:07,742 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/.tmp/family1/2b664b86344340ea9948a0e38151c04b as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/family1/2b664b86344340ea9948a0e38151c04b 2023-03-05 19:55:07,746 INFO [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in b9c878f961e0a32027f8960cfe26b4eb/family1 of b9c878f961e0a32027f8960cfe26b4eb into 2b664b86344340ea9948a0e38151c04b(size=192.0 K), total size for store is 192.0 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-05 19:55:07,746 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for b9c878f961e0a32027f8960cfe26b4eb: 2023-03-05 19:55:07,746 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678046094042.b9c878f961e0a32027f8960cfe26b4eb. 2023-03-05 19:55:07,746 INFO [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678046094042.b9c878f961e0a32027f8960cfe26b4eb., storeName=family1, fileCount=3, fileSize=388.1 K (190.5 K, 6.8 K, 190.9 K), priority=2, time=1678046106506; duration=1sec 2023-03-05 19:55:07,746 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678046094042.b9c878f961e0a32027f8960cfe26b4eb. 2023-03-05 19:55:07,746 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678046094042.b9c878f961e0a32027f8960cfe26b4eb. after waiting 0 ms 2023-03-05 19:55:07,746 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678046094042.b9c878f961e0a32027f8960cfe26b4eb. 2023-03-05 19:55:07,746 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-05 19:55:07,750 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678046094042.b9c878f961e0a32027f8960cfe26b4eb.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/family1/a1cf7b892fdf49569eaa81f22bfaf698, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/family1/23e83c6f75fa4c35bc4bb602947f83c9, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/family1/0fb0170e1b85486d91db4848320a4adc_SeqId_47_] to archive 2023-03-05 19:55:07,750 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678046094042.b9c878f961e0a32027f8960cfe26b4eb.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-03-05 19:55:07,752 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678046094042.b9c878f961e0a32027f8960cfe26b4eb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/family1/a1cf7b892fdf49569eaa81f22bfaf698 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/family1/a1cf7b892fdf49569eaa81f22bfaf698 2023-03-05 19:55:07,753 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678046094042.b9c878f961e0a32027f8960cfe26b4eb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/family1/23e83c6f75fa4c35bc4bb602947f83c9 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/family1/23e83c6f75fa4c35bc4bb602947f83c9 2023-03-05 19:55:07,754 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678046094042.b9c878f961e0a32027f8960cfe26b4eb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/family1/0fb0170e1b85486d91db4848320a4adc_SeqId_47_ to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/family1/0fb0170e1b85486d91db4848320a4adc_SeqId_47_ 2023-03-05 19:55:07,758 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/recovered.edits/50.seqid, newMaxSeqId=50, maxSeqId=1 2023-03-05 19:55:07,759 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678046094042.b9c878f961e0a32027f8960cfe26b4eb. 2023-03-05 19:55:07,759 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for b9c878f961e0a32027f8960cfe26b4eb: 2023-03-05 19:55:07,759 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3509): Not adding moved region record: b9c878f961e0a32027f8960cfe26b4eb to self. 2023-03-05 19:55:07,760 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed b9c878f961e0a32027f8960cfe26b4eb 2023-03-05 19:55:07,760 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=61 updating hbase:meta row=b9c878f961e0a32027f8960cfe26b4eb, regionState=CLOSED 2023-03-05 19:55:07,760 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678046094042.b9c878f961e0a32027f8960cfe26b4eb.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1678046107760"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678046107760"}]},"ts":"1678046107760"} 2023-03-05 19:55:07,763 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=62, resume processing ppid=61 2023-03-05 19:55:07,763 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=62, ppid=61, state=SUCCESS; CloseRegionProcedure b9c878f961e0a32027f8960cfe26b4eb, server=jenkins-hbase4.apache.org,37037,1678046018983 in 762 msec 2023-03-05 19:55:07,763 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=61, ppid=60, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=b9c878f961e0a32027f8960cfe26b4eb, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,37037,1678046018983; forceNewPlan=false, retain=true 2023-03-05 19:55:07,914 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=61 updating hbase:meta row=b9c878f961e0a32027f8960cfe26b4eb, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:07,914 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678046094042.b9c878f961e0a32027f8960cfe26b4eb.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1678046107914"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678046107914"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678046107914"}]},"ts":"1678046107914"} 2023-03-05 19:55:07,915 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=63, ppid=61, state=RUNNABLE; OpenRegionProcedure b9c878f961e0a32027f8960cfe26b4eb, server=jenkins-hbase4.apache.org,37037,1678046018983}] 2023-03-05 19:55:08,069 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678046094042.b9c878f961e0a32027f8960cfe26b4eb. 2023-03-05 19:55:08,070 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => b9c878f961e0a32027f8960cfe26b4eb, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678046094042.b9c878f961e0a32027f8960cfe26b4eb.', STARTKEY => '', ENDKEY => ''} 2023-03-05 19:55:08,070 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily b9c878f961e0a32027f8960cfe26b4eb 2023-03-05 19:55:08,070 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678046094042.b9c878f961e0a32027f8960cfe26b4eb.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-05 19:55:08,070 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for b9c878f961e0a32027f8960cfe26b4eb 2023-03-05 19:55:08,070 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for b9c878f961e0a32027f8960cfe26b4eb 2023-03-05 19:55:08,071 INFO [StoreOpener-b9c878f961e0a32027f8960cfe26b4eb-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region b9c878f961e0a32027f8960cfe26b4eb 2023-03-05 19:55:08,072 DEBUG [StoreOpener-b9c878f961e0a32027f8960cfe26b4eb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/family1 2023-03-05 19:55:08,072 DEBUG [StoreOpener-b9c878f961e0a32027f8960cfe26b4eb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/family1 2023-03-05 19:55:08,072 INFO [StoreOpener-b9c878f961e0a32027f8960cfe26b4eb-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b9c878f961e0a32027f8960cfe26b4eb columnFamilyName family1 2023-03-05 19:55:08,076 DEBUG [StoreOpener-b9c878f961e0a32027f8960cfe26b4eb-1] regionserver.HStore(539): loaded hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/family1/2b664b86344340ea9948a0e38151c04b 2023-03-05 19:55:08,077 INFO [StoreOpener-b9c878f961e0a32027f8960cfe26b4eb-1] regionserver.HStore(310): Store=b9c878f961e0a32027f8960cfe26b4eb/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-05 19:55:08,077 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb 2023-03-05 19:55:08,078 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb 2023-03-05 19:55:08,081 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for b9c878f961e0a32027f8960cfe26b4eb 2023-03-05 19:55:08,081 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened b9c878f961e0a32027f8960cfe26b4eb; next sequenceid=51; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69270435, jitterRate=0.03220991790294647}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-05 19:55:08,081 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for b9c878f961e0a32027f8960cfe26b4eb: 2023-03-05 19:55:08,082 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678046094042.b9c878f961e0a32027f8960cfe26b4eb., pid=63, masterSystemTime=1678046108067 2023-03-05 19:55:08,083 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678046094042.b9c878f961e0a32027f8960cfe26b4eb. 2023-03-05 19:55:08,083 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678046094042.b9c878f961e0a32027f8960cfe26b4eb. 2023-03-05 19:55:08,083 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=61 updating hbase:meta row=b9c878f961e0a32027f8960cfe26b4eb, regionState=OPEN, openSeqNum=51, regionLocation=jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:08,084 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678046094042.b9c878f961e0a32027f8960cfe26b4eb.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1678046108083"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678046108083"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678046108083"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678046108083"}]},"ts":"1678046108083"} 2023-03-05 19:55:08,086 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=63, resume processing ppid=61 2023-03-05 19:55:08,086 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=63, ppid=61, state=SUCCESS; OpenRegionProcedure b9c878f961e0a32027f8960cfe26b4eb, server=jenkins-hbase4.apache.org,37037,1678046018983 in 170 msec 2023-03-05 19:55:08,087 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=61, resume processing ppid=60 2023-03-05 19:55:08,087 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=61, ppid=60, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=b9c878f961e0a32027f8960cfe26b4eb, REOPEN/MOVE in 1.0890 sec 2023-03-05 19:55:08,089 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=60, resume processing ppid=59 2023-03-05 19:55:08,089 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=60, ppid=59, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 1.0930 sec 2023-03-05 19:55:08,090 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=59, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 1.5070 sec 2023-03-05 19:55:08,091 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1227): Checking to see if procedure is done pid=59 2023-03-05 19:55:08,092 INFO [Listener at localhost/40787] compactions.TestMobCompactor(278): alter status finished 2023-03-05 19:55:08,093 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=38221] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:08,094 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] procedure2.ProcedureExecutor(1029): Stored pid=64, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-03-05 19:55:08,095 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=64, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-03-05 19:55:08,095 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-05 19:55:08,095 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=64, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678046708095 2023-03-05 19:55:08,096 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-05 19:55:08,096 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 5 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column 'family1' 2023-03-05 19:55:08,097 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-05 19:55:08,097 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] zookeeper.ReadOnlyZKClient(139): Connect 0x2951c910 to 127.0.0.1:62957 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-05 19:55:08,101 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5cd4dca7, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-05 19:55:08,101 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-05 19:55:08,101 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-05 19:55:08,101 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-05 19:55:08,101 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-05 19:55:08,102 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-05 19:55:08,929 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128ddb0d0c4a79f423381b71e80ed4ea084 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151128ddb0d0c4a79f423381b71e80ed4ea084 2023-03-05 19:55:08,929 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-05 19:55:08,943 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-05 19:55:08,944 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:58972, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-05 19:55:08,951 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-05 19:55:08,952 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:43770, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-05 19:55:08,960 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101/family1/8b6dadf1912e489aa4955a4b3c70d382 first=Optional[r05] last=Optional[r8] 2023-03-05 19:55:08,961 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678046094042.b9c878f961e0a32027f8960cfe26b4eb., hostname=jenkins-hbase4.apache.org,37037,1678046018983, seqNum=51 for row with hfile group [{family1,hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101/family1/8b6dadf1912e489aa4955a4b3c70d382}] 2023-03-05 19:55:08,966 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(763): Validating hfile at hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101/family1/8b6dadf1912e489aa4955a4b3c70d382 for inclusion in b9c878f961e0a32027f8960cfe26b4eb/family1 2023-03-05 19:55:08,968 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(775): HFile bounds: first=r05 last=r8 2023-03-05 19:55:08,968 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(777): Region bounds: first= last= 2023-03-05 19:55:08,968 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HRegion(2446): Flush status journal for b9c878f961e0a32027f8960cfe26b4eb: 2023-03-05 19:55:08,969 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101/family1/8b6dadf1912e489aa4955a4b3c70d382 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__bj28nar1ndulkacclbqigf8c56qcrd6sdaclukl5ekaa4lmvfi14k2mdm0uoanpe/family1/8b6dadf1912e489aa4955a4b3c70d382 2023-03-05 19:55:08,971 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__bj28nar1ndulkacclbqigf8c56qcrd6sdaclukl5ekaa4lmvfi14k2mdm0uoanpe/family1/8b6dadf1912e489aa4955a4b3c70d382 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/family1/a76e924ecec741ef9efb46d64d074d37_SeqId_53_ 2023-03-05 19:55:08,971 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(849): Loaded HFile hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__bj28nar1ndulkacclbqigf8c56qcrd6sdaclukl5ekaa4lmvfi14k2mdm0uoanpe/family1/8b6dadf1912e489aa4955a4b3c70d382 into b9c878f961e0a32027f8960cfe26b4eb/family1 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/family1/a76e924ecec741ef9efb46d64d074d37_SeqId_53_ - updating store file list. 2023-03-05 19:55:08,975 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(882): Loaded HFile hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/family1/a76e924ecec741ef9efb46d64d074d37_SeqId_53_ into b9c878f961e0a32027f8960cfe26b4eb/family1 2023-03-05 19:55:08,975 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(855): Successfully loaded hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__bj28nar1ndulkacclbqigf8c56qcrd6sdaclukl5ekaa4lmvfi14k2mdm0uoanpe/family1/8b6dadf1912e489aa4955a4b3c70d382 into b9c878f961e0a32027f8960cfe26b4eb/family1 (new location: hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/family1/a76e924ecec741ef9efb46d64d074d37_SeqId_53_) 2023-03-05 19:55:08,975 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__bj28nar1ndulkacclbqigf8c56qcrd6sdaclukl5ekaa4lmvfi14k2mdm0uoanpe/family1/8b6dadf1912e489aa4955a4b3c70d382 2023-03-05 19:55:08,976 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-05 19:55:08,977 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-05 19:55:08,978 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151120138a53602a1249ef9362f50f52690232 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151120138a53602a1249ef9362f50f52690232 2023-03-05 19:55:08,979 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201511288cf82d5dd37a492784e0be184bdce971 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201511288cf82d5dd37a492784e0be184bdce971 2023-03-05 19:55:08,979 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2 expired mob files are deleted 2023-03-05 19:55:08,979 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-03-05 19:55:08,980 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-05 19:55:08,980 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-05 19:55:08,980 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-05 19:55:08,980 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-05 19:55:08,980 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-05 19:55:08,981 DEBUG [PEWorker-4] locking.LockProcedure(242): UNLOCKED pid=64, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-03-05 19:55:08,982 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=64, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE in 887 msec 2023-03-05 19:55:09,025 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.HMaster$12(2454): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily from 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'DAILY'} 2023-03-05 19:55:09,025 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure2.ProcedureExecutor(1029): Stored pid=65, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:09,433 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000003 2023-03-05 19:55:09,434 DEBUG [PEWorker-1] util.FSTableDescriptors(519): Deleted hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000002 2023-03-05 19:55:09,434 INFO [PEWorker-1] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000003 2023-03-05 19:55:09,436 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=66, ppid=65, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily}] 2023-03-05 19:55:09,438 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=67, ppid=66, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=b9c878f961e0a32027f8960cfe26b4eb, REOPEN/MOVE}] 2023-03-05 19:55:09,438 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=67, ppid=66, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=b9c878f961e0a32027f8960cfe26b4eb, REOPEN/MOVE 2023-03-05 19:55:09,439 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=67 updating hbase:meta row=b9c878f961e0a32027f8960cfe26b4eb, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:09,439 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678046094042.b9c878f961e0a32027f8960cfe26b4eb.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1678046109439"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678046109439"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678046109439"}]},"ts":"1678046109439"} 2023-03-05 19:55:09,440 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=68, ppid=67, state=RUNNABLE; CloseRegionProcedure b9c878f961e0a32027f8960cfe26b4eb, server=jenkins-hbase4.apache.org,37037,1678046018983}] 2023-03-05 19:55:09,592 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close b9c878f961e0a32027f8960cfe26b4eb 2023-03-05 19:55:09,592 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing b9c878f961e0a32027f8960cfe26b4eb, disabling compactions & flushes 2023-03-05 19:55:09,592 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678046094042.b9c878f961e0a32027f8960cfe26b4eb. 2023-03-05 19:55:09,592 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678046094042.b9c878f961e0a32027f8960cfe26b4eb. 2023-03-05 19:55:09,592 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678046094042.b9c878f961e0a32027f8960cfe26b4eb. after waiting 0 ms 2023-03-05 19:55:09,592 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678046094042.b9c878f961e0a32027f8960cfe26b4eb. 2023-03-05 19:55:09,598 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/recovered.edits/55.seqid, newMaxSeqId=55, maxSeqId=50 2023-03-05 19:55:09,599 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678046094042.b9c878f961e0a32027f8960cfe26b4eb. 2023-03-05 19:55:09,599 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for b9c878f961e0a32027f8960cfe26b4eb: 2023-03-05 19:55:09,599 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3509): Not adding moved region record: b9c878f961e0a32027f8960cfe26b4eb to self. 2023-03-05 19:55:09,600 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed b9c878f961e0a32027f8960cfe26b4eb 2023-03-05 19:55:09,601 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=67 updating hbase:meta row=b9c878f961e0a32027f8960cfe26b4eb, regionState=CLOSED 2023-03-05 19:55:09,601 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678046094042.b9c878f961e0a32027f8960cfe26b4eb.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1678046109601"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678046109601"}]},"ts":"1678046109601"} 2023-03-05 19:55:09,602 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=68, resume processing ppid=67 2023-03-05 19:55:09,603 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=68, ppid=67, state=SUCCESS; CloseRegionProcedure b9c878f961e0a32027f8960cfe26b4eb, server=jenkins-hbase4.apache.org,37037,1678046018983 in 162 msec 2023-03-05 19:55:09,603 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=67, ppid=66, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=b9c878f961e0a32027f8960cfe26b4eb, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,37037,1678046018983; forceNewPlan=false, retain=true 2023-03-05 19:55:09,753 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=67 updating hbase:meta row=b9c878f961e0a32027f8960cfe26b4eb, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:09,753 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678046094042.b9c878f961e0a32027f8960cfe26b4eb.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1678046109753"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678046109753"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678046109753"}]},"ts":"1678046109753"} 2023-03-05 19:55:09,755 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=69, ppid=67, state=RUNNABLE; OpenRegionProcedure b9c878f961e0a32027f8960cfe26b4eb, server=jenkins-hbase4.apache.org,37037,1678046018983}] 2023-03-05 19:55:09,909 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678046094042.b9c878f961e0a32027f8960cfe26b4eb. 2023-03-05 19:55:09,909 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => b9c878f961e0a32027f8960cfe26b4eb, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678046094042.b9c878f961e0a32027f8960cfe26b4eb.', STARTKEY => '', ENDKEY => ''} 2023-03-05 19:55:09,909 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily b9c878f961e0a32027f8960cfe26b4eb 2023-03-05 19:55:09,909 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678046094042.b9c878f961e0a32027f8960cfe26b4eb.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-05 19:55:09,909 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for b9c878f961e0a32027f8960cfe26b4eb 2023-03-05 19:55:09,909 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for b9c878f961e0a32027f8960cfe26b4eb 2023-03-05 19:55:09,910 INFO [StoreOpener-b9c878f961e0a32027f8960cfe26b4eb-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region b9c878f961e0a32027f8960cfe26b4eb 2023-03-05 19:55:09,911 DEBUG [StoreOpener-b9c878f961e0a32027f8960cfe26b4eb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/family1 2023-03-05 19:55:09,911 DEBUG [StoreOpener-b9c878f961e0a32027f8960cfe26b4eb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/family1 2023-03-05 19:55:09,911 INFO [StoreOpener-b9c878f961e0a32027f8960cfe26b4eb-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b9c878f961e0a32027f8960cfe26b4eb columnFamilyName family1 2023-03-05 19:55:09,916 DEBUG [StoreOpener-b9c878f961e0a32027f8960cfe26b4eb-1] regionserver.HStore(539): loaded hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/family1/2b664b86344340ea9948a0e38151c04b 2023-03-05 19:55:09,919 DEBUG [StoreOpener-b9c878f961e0a32027f8960cfe26b4eb-1] regionserver.HStore(539): loaded hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/family1/a76e924ecec741ef9efb46d64d074d37_SeqId_53_ 2023-03-05 19:55:09,919 INFO [StoreOpener-b9c878f961e0a32027f8960cfe26b4eb-1] regionserver.HStore(310): Store=b9c878f961e0a32027f8960cfe26b4eb/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-05 19:55:09,920 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb 2023-03-05 19:55:09,920 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb 2023-03-05 19:55:09,923 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for b9c878f961e0a32027f8960cfe26b4eb 2023-03-05 19:55:09,923 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened b9c878f961e0a32027f8960cfe26b4eb; next sequenceid=56; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59829714, jitterRate=-0.10846778750419617}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-05 19:55:09,923 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for b9c878f961e0a32027f8960cfe26b4eb: 2023-03-05 19:55:09,924 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678046094042.b9c878f961e0a32027f8960cfe26b4eb., pid=69, masterSystemTime=1678046109906 2023-03-05 19:55:09,925 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678046094042.b9c878f961e0a32027f8960cfe26b4eb. 2023-03-05 19:55:09,925 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678046094042.b9c878f961e0a32027f8960cfe26b4eb. 2023-03-05 19:55:09,925 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=67 updating hbase:meta row=b9c878f961e0a32027f8960cfe26b4eb, regionState=OPEN, openSeqNum=56, regionLocation=jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:09,925 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678046094042.b9c878f961e0a32027f8960cfe26b4eb.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1678046109925"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678046109925"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678046109925"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678046109925"}]},"ts":"1678046109925"} 2023-03-05 19:55:09,927 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=69, resume processing ppid=67 2023-03-05 19:55:09,927 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=69, ppid=67, state=SUCCESS; OpenRegionProcedure b9c878f961e0a32027f8960cfe26b4eb, server=jenkins-hbase4.apache.org,37037,1678046018983 in 171 msec 2023-03-05 19:55:09,929 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=67, resume processing ppid=66 2023-03-05 19:55:09,929 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=67, ppid=66, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=b9c878f961e0a32027f8960cfe26b4eb, REOPEN/MOVE in 489 msec 2023-03-05 19:55:09,930 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=66, resume processing ppid=65 2023-03-05 19:55:09,930 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=66, ppid=65, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 493 msec 2023-03-05 19:55:09,931 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=65, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 905 msec 2023-03-05 19:55:09,932 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1227): Checking to see if procedure is done pid=65 2023-03-05 19:55:09,932 INFO [Listener at localhost/40787] compactions.TestMobCompactor(278): alter status finished 2023-03-05 19:55:09,933 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=38221] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'DAILY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-05 19:55:09,934 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] procedure2.ProcedureExecutor(1029): Stored pid=70, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-03-05 19:55:09,935 DEBUG [PEWorker-5] locking.LockProcedure(309): LOCKED pid=70, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-03-05 19:55:09,935 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-05 19:55:09,935 INFO [PEWorker-5] procedure2.TimeoutExecutorThread(81): ADDED pid=70, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678046709935 2023-03-05 19:55:09,936 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-05 19:55:09,936 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 4 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column 'family1' 2023-03-05 19:55:09,936 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-05 19:55:09,937 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] zookeeper.ReadOnlyZKClient(139): Connect 0x46149275 to 127.0.0.1:62957 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-05 19:55:09,941 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@678446f4, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-05 19:55:09,941 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151128 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-05 19:55:09,941 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151128 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-05 19:55:09,941 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151205 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-05 19:55:09,941 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151205 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-05 19:55:09,941 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160103 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-05 19:55:09,941 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160103 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-05 19:55:09,941 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-05 19:55:09,941 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-05 19:55:09,941 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-05 19:55:09,941 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-05 19:55:09,942 DEBUG [PEWorker-3] locking.LockProcedure(242): UNLOCKED pid=70, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-03-05 19:55:09,943 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=70, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE in 8 msec 2023-03-05 19:55:09,986 INFO [Listener at localhost/40787] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily Thread=240 (was 248), OpenFileDescriptor=620 (was 624), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=152 (was 195), ProcessCount=173 (was 173), AvailableMemoryMB=4674 (was 4736) 2023-03-05 19:55:09,997 INFO [Listener at localhost/40787] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testGetAfterCompaction Thread=240, OpenFileDescriptor=620, MaxFileDescriptor=60000, SystemLoadAverage=152, ProcessCount=173, AvailableMemoryMB=4673 2023-03-05 19:55:09,999 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testGetAfterCompaction', {TABLE_ATTRIBUTES => {coprocessor$1 => '|org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro|1073741823|'}}, {NAME => 'f1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-03-05 19:55:10,000 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure2.ProcedureExecutor(1029): Stored pid=71, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testGetAfterCompaction 2023-03-05 19:55:10,001 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_PRE_OPERATION 2023-03-05 19:55:10,001 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testGetAfterCompaction" procId is: 71 2023-03-05 19:55:10,002 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-05 19:55:10,002 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-03-05 19:55:10,003 DEBUG [HFileArchiver-11] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/.tmp/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746 2023-03-05 19:55:10,003 DEBUG [HFileArchiver-11] backup.HFileArchiver(153): Directory hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/.tmp/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746 empty. 2023-03-05 19:55:10,004 DEBUG [HFileArchiver-11] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/.tmp/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746 2023-03-05 19:55:10,004 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived testGetAfterCompaction regions 2023-03-05 19:55:10,103 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-03-05 19:55:10,304 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-03-05 19:55:10,411 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/.tmp/data/default/testGetAfterCompaction/.tabledesc/.tableinfo.0000000001 2023-03-05 19:55:10,412 INFO [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(7675): creating {ENCODED => 2f859dd19bfc7b5149ac61d5410de746, NAME => 'testGetAfterCompaction,,1678046109999.2f859dd19bfc7b5149ac61d5410de746.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testGetAfterCompaction', {TABLE_ATTRIBUTES => {coprocessor$1 => '|org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro|1073741823|'}}, {NAME => 'f1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/.tmp 2023-03-05 19:55:10,605 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-03-05 19:55:10,818 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(866): Instantiated testGetAfterCompaction,,1678046109999.2f859dd19bfc7b5149ac61d5410de746.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-05 19:55:10,818 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1604): Closing 2f859dd19bfc7b5149ac61d5410de746, disabling compactions & flushes 2023-03-05 19:55:10,818 INFO [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1626): Closing region testGetAfterCompaction,,1678046109999.2f859dd19bfc7b5149ac61d5410de746. 2023-03-05 19:55:10,818 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testGetAfterCompaction,,1678046109999.2f859dd19bfc7b5149ac61d5410de746. 2023-03-05 19:55:10,818 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1714): Acquired close lock on testGetAfterCompaction,,1678046109999.2f859dd19bfc7b5149ac61d5410de746. after waiting 0 ms 2023-03-05 19:55:10,818 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1724): Updates disabled for region testGetAfterCompaction,,1678046109999.2f859dd19bfc7b5149ac61d5410de746. 2023-03-05 19:55:10,818 INFO [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1838): Closed testGetAfterCompaction,,1678046109999.2f859dd19bfc7b5149ac61d5410de746. 2023-03-05 19:55:10,818 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1558): Region close journal for 2f859dd19bfc7b5149ac61d5410de746: 2023-03-05 19:55:10,820 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_ADD_TO_META 2023-03-05 19:55:10,821 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testGetAfterCompaction,,1678046109999.2f859dd19bfc7b5149ac61d5410de746.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1678046110821"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678046110821"}]},"ts":"1678046110821"} 2023-03-05 19:55:10,822 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-03-05 19:55:10,822 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-05 19:55:10,823 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678046110823"}]},"ts":"1678046110823"} 2023-03-05 19:55:10,823 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testGetAfterCompaction, state=ENABLING in hbase:meta 2023-03-05 19:55:10,826 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=72, ppid=71, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testGetAfterCompaction, region=2f859dd19bfc7b5149ac61d5410de746, ASSIGN}] 2023-03-05 19:55:10,827 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, ppid=71, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testGetAfterCompaction, region=2f859dd19bfc7b5149ac61d5410de746, ASSIGN 2023-03-05 19:55:10,828 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=72, ppid=71, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testGetAfterCompaction, region=2f859dd19bfc7b5149ac61d5410de746, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,37037,1678046018983; forceNewPlan=false, retain=false 2023-03-05 19:55:10,979 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=72 updating hbase:meta row=2f859dd19bfc7b5149ac61d5410de746, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:10,979 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testGetAfterCompaction,,1678046109999.2f859dd19bfc7b5149ac61d5410de746.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1678046110979"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678046110979"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678046110979"}]},"ts":"1678046110979"} 2023-03-05 19:55:10,980 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=73, ppid=72, state=RUNNABLE; OpenRegionProcedure 2f859dd19bfc7b5149ac61d5410de746, server=jenkins-hbase4.apache.org,37037,1678046018983}] 2023-03-05 19:55:11,105 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-03-05 19:55:11,134 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testGetAfterCompaction,,1678046109999.2f859dd19bfc7b5149ac61d5410de746. 2023-03-05 19:55:11,134 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 2f859dd19bfc7b5149ac61d5410de746, NAME => 'testGetAfterCompaction,,1678046109999.2f859dd19bfc7b5149ac61d5410de746.', STARTKEY => '', ENDKEY => ''} 2023-03-05 19:55:11,134 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro with path null and priority 1073741823 2023-03-05 19:55:11,136 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro from HTD of testGetAfterCompaction successfully. 2023-03-05 19:55:11,136 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testGetAfterCompaction 2f859dd19bfc7b5149ac61d5410de746 2023-03-05 19:55:11,136 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testGetAfterCompaction,,1678046109999.2f859dd19bfc7b5149ac61d5410de746.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-05 19:55:11,136 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 2f859dd19bfc7b5149ac61d5410de746 2023-03-05 19:55:11,136 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 2f859dd19bfc7b5149ac61d5410de746 2023-03-05 19:55:11,137 INFO [StoreOpener-2f859dd19bfc7b5149ac61d5410de746-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family f1 of region 2f859dd19bfc7b5149ac61d5410de746 2023-03-05 19:55:11,138 DEBUG [StoreOpener-2f859dd19bfc7b5149ac61d5410de746-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/f1 2023-03-05 19:55:11,138 DEBUG [StoreOpener-2f859dd19bfc7b5149ac61d5410de746-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/f1 2023-03-05 19:55:11,139 INFO [StoreOpener-2f859dd19bfc7b5149ac61d5410de746-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2f859dd19bfc7b5149ac61d5410de746 columnFamilyName f1 2023-03-05 19:55:11,140 INFO [StoreOpener-2f859dd19bfc7b5149ac61d5410de746-1] regionserver.HStore(310): Store=2f859dd19bfc7b5149ac61d5410de746/f1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-05 19:55:11,141 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746 2023-03-05 19:55:11,141 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746 2023-03-05 19:55:11,143 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 2f859dd19bfc7b5149ac61d5410de746 2023-03-05 19:55:11,145 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-05 19:55:11,145 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 2f859dd19bfc7b5149ac61d5410de746; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66700412, jitterRate=-0.006086409091949463}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-05 19:55:11,145 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 2f859dd19bfc7b5149ac61d5410de746: 2023-03-05 19:55:11,146 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testGetAfterCompaction,,1678046109999.2f859dd19bfc7b5149ac61d5410de746., pid=73, masterSystemTime=1678046111131 2023-03-05 19:55:11,147 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testGetAfterCompaction,,1678046109999.2f859dd19bfc7b5149ac61d5410de746. 2023-03-05 19:55:11,147 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testGetAfterCompaction,,1678046109999.2f859dd19bfc7b5149ac61d5410de746. 2023-03-05 19:55:11,147 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=72 updating hbase:meta row=2f859dd19bfc7b5149ac61d5410de746, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:11,147 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testGetAfterCompaction,,1678046109999.2f859dd19bfc7b5149ac61d5410de746.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1678046111147"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678046111147"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678046111147"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678046111147"}]},"ts":"1678046111147"} 2023-03-05 19:55:11,150 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=73, resume processing ppid=72 2023-03-05 19:55:11,150 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=73, ppid=72, state=SUCCESS; OpenRegionProcedure 2f859dd19bfc7b5149ac61d5410de746, server=jenkins-hbase4.apache.org,37037,1678046018983 in 168 msec 2023-03-05 19:55:11,151 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=72, resume processing ppid=71 2023-03-05 19:55:11,151 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=72, ppid=71, state=SUCCESS; TransitRegionStateProcedure table=testGetAfterCompaction, region=2f859dd19bfc7b5149ac61d5410de746, ASSIGN in 324 msec 2023-03-05 19:55:11,151 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-05 19:55:11,152 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678046111151"}]},"ts":"1678046111151"} 2023-03-05 19:55:11,152 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testGetAfterCompaction, state=ENABLED in hbase:meta 2023-03-05 19:55:11,154 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_POST_OPERATION 2023-03-05 19:55:11,155 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=71, state=SUCCESS; CreateTableProcedure table=testGetAfterCompaction in 1.1540 sec 2023-03-05 19:55:11,172 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testGetAfterCompaction' 2023-03-05 19:55:12,106 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-03-05 19:55:12,107 INFO [Listener at localhost/40787] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testGetAfterCompaction, procId: 71 completed 2023-03-05 19:55:12,107 DEBUG [Listener at localhost/40787] hbase.HBaseTestingUtility(3430): Waiting until all regions of table testGetAfterCompaction get assigned. Timeout = 60000ms 2023-03-05 19:55:12,109 INFO [Listener at localhost/40787] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-03-05 19:55:12,112 INFO [Listener at localhost/40787] hbase.HBaseTestingUtility(3484): All regions for table testGetAfterCompaction assigned to meta. Checking AM states. 2023-03-05 19:55:12,112 INFO [Listener at localhost/40787] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-03-05 19:55:12,112 INFO [Listener at localhost/40787] hbase.HBaseTestingUtility(3504): All regions for table testGetAfterCompaction assigned. 2023-03-05 19:55:12,116 INFO [Listener at localhost/40787] regionserver.HRegion(2745): Flushing 2f859dd19bfc7b5149ac61d5410de746 1/1 column families, dataSize=37 B heapSize=368 B 2023-03-05 19:55:12,528 INFO [Listener at localhost/40787] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303055b1784fb29244d4bb9cec3e647f8c119 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e202303055b1784fb29244d4bb9cec3e647f8c119 2023-03-05 19:55:12,933 INFO [Listener at localhost/40787] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=37, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/.tmp/f1/a84cae38d2f449e7aca1dd304e2c3d77 2023-03-05 19:55:12,938 DEBUG [Listener at localhost/40787] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/.tmp/f1/a84cae38d2f449e7aca1dd304e2c3d77 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/f1/a84cae38d2f449e7aca1dd304e2c3d77 2023-03-05 19:55:12,942 INFO [Listener at localhost/40787] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/f1/a84cae38d2f449e7aca1dd304e2c3d77, entries=1, sequenceid=5, filesize=4.9 K 2023-03-05 19:55:12,943 INFO [Listener at localhost/40787] regionserver.HRegion(2948): Finished flush of dataSize ~37 B/37, heapSize ~352 B/352, currentSize=0 B/0 for 2f859dd19bfc7b5149ac61d5410de746 in 827ms, sequenceid=5, compaction requested=false 2023-03-05 19:55:12,944 DEBUG [Listener at localhost/40787] regionserver.HRegion(2446): Flush status journal for 2f859dd19bfc7b5149ac61d5410de746: 2023-03-05 19:55:12,945 INFO [Listener at localhost/40787] regionserver.HRegion(2745): Flushing 2f859dd19bfc7b5149ac61d5410de746 1/1 column families, dataSize=37 B heapSize=368 B 2023-03-05 19:55:13,295 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-05 19:55:13,357 INFO [Listener at localhost/40787] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023030540c0ec7a30d845be9dd4c532f9bc14f5 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e2023030540c0ec7a30d845be9dd4c532f9bc14f5 2023-03-05 19:55:13,762 INFO [Listener at localhost/40787] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=37, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/.tmp/f1/387dcc6a6b6a4931bad7bcf5ecd7ac99 2023-03-05 19:55:13,766 DEBUG [Listener at localhost/40787] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/.tmp/f1/387dcc6a6b6a4931bad7bcf5ecd7ac99 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/f1/387dcc6a6b6a4931bad7bcf5ecd7ac99 2023-03-05 19:55:13,769 INFO [Listener at localhost/40787] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/f1/387dcc6a6b6a4931bad7bcf5ecd7ac99, entries=1, sequenceid=9, filesize=4.9 K 2023-03-05 19:55:13,770 INFO [Listener at localhost/40787] regionserver.HRegion(2948): Finished flush of dataSize ~37 B/37, heapSize ~352 B/352, currentSize=0 B/0 for 2f859dd19bfc7b5149ac61d5410de746 in 825ms, sequenceid=9, compaction requested=false 2023-03-05 19:55:13,770 DEBUG [Listener at localhost/40787] regionserver.HRegion(2446): Flush status journal for 2f859dd19bfc7b5149ac61d5410de746: 2023-03-05 19:55:13,771 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=38221] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'f1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}] of the table testGetAfterCompaction 2023-03-05 19:55:13,772 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] procedure2.ProcedureExecutor(1029): Stored pid=74, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE 2023-03-05 19:55:13,773 DEBUG [PEWorker-5] locking.LockProcedure(309): LOCKED pid=74, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE 2023-03-05 19:55:13,774 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] mob.MobUtils(833): start MOB compaction of files for table='testGetAfterCompaction', column='f1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-05 19:55:13,774 INFO [PEWorker-5] procedure2.TimeoutExecutorThread(81): ADDED pid=74, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678046713774 2023-03-05 19:55:13,774 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-03-05 19:55:13,774 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 2 selected files, and 0 irrelevant files table 'testGetAfterCompaction' and column 'f1' 2023-03-05 19:55:13,774 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testGetAfterCompaction' column='f1' 2023-03-05 19:55:13,775 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] zookeeper.ReadOnlyZKClient(139): Connect 0x21ece7aa to 127.0.0.1:62957 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-05 19:55:13,779 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@641ab685, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-05 19:55:13,780 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230305 for table='testGetAfterCompaction' and column='f1' 2023-03-05 19:55:13,823 INFO [Listener at localhost/40787] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/f1/387dcc6a6b6a4931bad7bcf5ecd7ac99; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678046113762; access_time=1678046112947; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-05 19:55:13,823 INFO [Listener at localhost/40787] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/f1/a84cae38d2f449e7aca1dd304e2c3d77; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678046112933; access_time=1678046112119; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-05 19:55:13,873 INFO [Listener at localhost/40787] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/f1/387dcc6a6b6a4931bad7bcf5ecd7ac99; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678046113762; access_time=1678046112947; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-05 19:55:13,874 INFO [Listener at localhost/40787] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/f1/a84cae38d2f449e7aca1dd304e2c3d77; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678046112933; access_time=1678046112119; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-05 19:55:13,924 INFO [Listener at localhost/40787] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/f1/387dcc6a6b6a4931bad7bcf5ecd7ac99; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678046113762; access_time=1678046112947; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-05 19:55:13,924 INFO [Listener at localhost/40787] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/f1/a84cae38d2f449e7aca1dd304e2c3d77; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678046112933; access_time=1678046112119; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-05 19:55:13,975 INFO [Listener at localhost/40787] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/f1/387dcc6a6b6a4931bad7bcf5ecd7ac99; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678046113762; access_time=1678046112947; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-05 19:55:13,975 INFO [Listener at localhost/40787] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/f1/a84cae38d2f449e7aca1dd304e2c3d77; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678046112933; access_time=1678046112119; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-05 19:55:14,026 INFO [Listener at localhost/40787] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/f1/387dcc6a6b6a4931bad7bcf5ecd7ac99; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678046113762; access_time=1678046112947; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-05 19:55:14,026 INFO [Listener at localhost/40787] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/f1/a84cae38d2f449e7aca1dd304e2c3d77; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678046112933; access_time=1678046112119; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-05 19:55:14,076 INFO [Listener at localhost/40787] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/f1/387dcc6a6b6a4931bad7bcf5ecd7ac99; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678046113762; access_time=1678046112947; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-05 19:55:14,077 INFO [Listener at localhost/40787] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/f1/a84cae38d2f449e7aca1dd304e2c3d77; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678046112933; access_time=1678046112119; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-05 19:55:14,127 INFO [Listener at localhost/40787] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/f1/387dcc6a6b6a4931bad7bcf5ecd7ac99; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678046113762; access_time=1678046112947; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-05 19:55:14,128 INFO [Listener at localhost/40787] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/f1/a84cae38d2f449e7aca1dd304e2c3d77; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678046112933; access_time=1678046112119; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-05 19:55:14,178 INFO [Listener at localhost/40787] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/f1/387dcc6a6b6a4931bad7bcf5ecd7ac99; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678046113762; access_time=1678046112947; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-05 19:55:14,178 INFO [Listener at localhost/40787] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/f1/a84cae38d2f449e7aca1dd304e2c3d77; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678046112933; access_time=1678046112119; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-05 19:55:14,229 INFO [Listener at localhost/40787] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/f1/387dcc6a6b6a4931bad7bcf5ecd7ac99; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678046113762; access_time=1678046112947; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-05 19:55:14,229 INFO [Listener at localhost/40787] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/f1/a84cae38d2f449e7aca1dd304e2c3d77; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678046112933; access_time=1678046112119; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-05 19:55:14,280 INFO [Listener at localhost/40787] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/f1/387dcc6a6b6a4931bad7bcf5ecd7ac99; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678046113762; access_time=1678046112947; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-05 19:55:14,280 INFO [Listener at localhost/40787] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/f1/a84cae38d2f449e7aca1dd304e2c3d77; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678046112933; access_time=1678046112119; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-05 19:55:14,331 INFO [Listener at localhost/40787] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/f1/387dcc6a6b6a4931bad7bcf5ecd7ac99; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678046113762; access_time=1678046112947; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-05 19:55:14,331 INFO [Listener at localhost/40787] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/f1/a84cae38d2f449e7aca1dd304e2c3d77; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678046112933; access_time=1678046112119; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-05 19:55:14,382 INFO [Listener at localhost/40787] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/f1/387dcc6a6b6a4931bad7bcf5ecd7ac99; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678046113762; access_time=1678046112947; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-05 19:55:14,382 INFO [Listener at localhost/40787] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/f1/a84cae38d2f449e7aca1dd304e2c3d77; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678046112933; access_time=1678046112119; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-05 19:55:14,433 INFO [Listener at localhost/40787] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/f1/387dcc6a6b6a4931bad7bcf5ecd7ac99; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678046113762; access_time=1678046112947; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-05 19:55:14,433 INFO [Listener at localhost/40787] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/f1/a84cae38d2f449e7aca1dd304e2c3d77; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678046112933; access_time=1678046112119; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-05 19:55:14,483 INFO [Listener at localhost/40787] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/f1/387dcc6a6b6a4931bad7bcf5ecd7ac99; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678046113762; access_time=1678046112947; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-05 19:55:14,483 INFO [Listener at localhost/40787] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/f1/a84cae38d2f449e7aca1dd304e2c3d77; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678046112933; access_time=1678046112119; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-05 19:55:14,534 INFO [Listener at localhost/40787] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/f1/387dcc6a6b6a4931bad7bcf5ecd7ac99; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678046113762; access_time=1678046112947; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-05 19:55:14,534 INFO [Listener at localhost/40787] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/f1/a84cae38d2f449e7aca1dd304e2c3d77; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678046112933; access_time=1678046112119; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-05 19:55:14,585 INFO [Listener at localhost/40787] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/f1/387dcc6a6b6a4931bad7bcf5ecd7ac99; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678046113762; access_time=1678046112947; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-05 19:55:14,585 INFO [Listener at localhost/40787] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/f1/a84cae38d2f449e7aca1dd304e2c3d77; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678046112933; access_time=1678046112119; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-05 19:55:14,600 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303054c4d0b51776d4494b98abd70817af3e1 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e202303054c4d0b51776d4494b98abd70817af3e1 2023-03-05 19:55:14,601 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230305 table='testGetAfterCompaction' column='f1' 2023-03-05 19:55:14,613 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-05 19:55:14,614 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:37690, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-05 19:55:14,620 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-05 19:55:14,621 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:56542, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-05 19:55:14,629 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230305/f1/e65d8045937a40af82831ab7f9ad43fa first=Optional[r1] last=Optional[r2] 2023-03-05 19:55:14,630 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testGetAfterCompaction,,1678046109999.2f859dd19bfc7b5149ac61d5410de746., hostname=jenkins-hbase4.apache.org,37037,1678046018983, seqNum=2 for row with hfile group [{f1,hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230305/f1/e65d8045937a40af82831ab7f9ad43fa}] 2023-03-05 19:55:14,633 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(763): Validating hfile at hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230305/f1/e65d8045937a40af82831ab7f9ad43fa for inclusion in 2f859dd19bfc7b5149ac61d5410de746/f1 2023-03-05 19:55:14,636 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(775): HFile bounds: first=r1 last=r2 2023-03-05 19:55:14,636 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(777): Region bounds: first= last= 2023-03-05 19:55:14,636 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HRegion(2446): Flush status journal for 2f859dd19bfc7b5149ac61d5410de746: 2023-03-05 19:55:14,636 INFO [Listener at localhost/40787] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/f1/387dcc6a6b6a4931bad7bcf5ecd7ac99; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678046113762; access_time=1678046112947; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-05 19:55:14,636 INFO [Listener at localhost/40787] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/f1/a84cae38d2f449e7aca1dd304e2c3d77; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678046112933; access_time=1678046112119; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-05 19:55:14,637 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230305/f1/e65d8045937a40af82831ab7f9ad43fa to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testGetAfterCompaction__93sbrj4a11gkk07dkoug80gsc7abhei4672pgiact5tdulrnef291iosji06ovm2/f1/e65d8045937a40af82831ab7f9ad43fa 2023-03-05 19:55:14,639 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testGetAfterCompaction__93sbrj4a11gkk07dkoug80gsc7abhei4672pgiact5tdulrnef291iosji06ovm2/f1/e65d8045937a40af82831ab7f9ad43fa as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/f1/71b00fd6c6a9400bb51d628286b49cac_SeqId_12_ 2023-03-05 19:55:14,639 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(849): Loaded HFile hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testGetAfterCompaction__93sbrj4a11gkk07dkoug80gsc7abhei4672pgiact5tdulrnef291iosji06ovm2/f1/e65d8045937a40af82831ab7f9ad43fa into 2f859dd19bfc7b5149ac61d5410de746/f1 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/f1/71b00fd6c6a9400bb51d628286b49cac_SeqId_12_ - updating store file list. 2023-03-05 19:55:14,642 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(882): Loaded HFile hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/f1/71b00fd6c6a9400bb51d628286b49cac_SeqId_12_ into 2f859dd19bfc7b5149ac61d5410de746/f1 2023-03-05 19:55:14,643 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(855): Successfully loaded hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testGetAfterCompaction__93sbrj4a11gkk07dkoug80gsc7abhei4672pgiact5tdulrnef291iosji06ovm2/f1/e65d8045937a40af82831ab7f9ad43fa into 2f859dd19bfc7b5149ac61d5410de746/f1 (new location: hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/f1/71b00fd6c6a9400bb51d628286b49cac_SeqId_12_) 2023-03-05 19:55:14,643 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testGetAfterCompaction__93sbrj4a11gkk07dkoug80gsc7abhei4672pgiact5tdulrnef291iosji06ovm2/f1/e65d8045937a40af82831ab7f9ad43fa 2023-03-05 19:55:14,643 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(1912): 2f859dd19bfc7b5149ac61d5410de746/f1 is initiating minor compaction 2023-03-05 19:55:14,643 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@2f0cc817; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-05 19:55:14,643 INFO [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 2f859dd19bfc7b5149ac61d5410de746/f1 in testGetAfterCompaction,,1678046109999.2f859dd19bfc7b5149ac61d5410de746. 2023-03-05 19:55:14,644 INFO [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/f1/387dcc6a6b6a4931bad7bcf5ecd7ac99, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/f1/71b00fd6c6a9400bb51d628286b49cac_SeqId_12_] into tmpdir=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/.tmp, totalSize=10.0 K 2023-03-05 19:55:14,644 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.Compactor(207): Compacting 387dcc6a6b6a4931bad7bcf5ecd7ac99, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=9 2023-03-05 19:55:14,644 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.Compactor(207): Compacting 71b00fd6c6a9400bb51d628286b49cac_SeqId_12_, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13 2023-03-05 19:55:14,645 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230305 table='testGetAfterCompaction' column='f1' 2023-03-05 19:55:14,646 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-05 19:55:14,648 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e2023030540c0ec7a30d845be9dd4c532f9bc14f5 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e2023030540c0ec7a30d845be9dd4c532f9bc14f5 2023-03-05 19:55:14,649 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e202303055b1784fb29244d4bb9cec3e647f8c119 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e202303055b1784fb29244d4bb9cec3e647f8c119 2023-03-05 19:55:14,649 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testGetAfterCompaction 2 expired mob files are deleted 2023-03-05 19:55:14,649 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230305' 2023-03-05 19:55:14,650 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230305 for table='testGetAfterCompaction' and column='f1' 2023-03-05 19:55:14,650 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] compactions.PartitionedMobCompactor(354): After compaction, there are 1 mob files. table='testGetAfterCompaction' column='f1' 2023-03-05 19:55:14,650 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] mob.MobUtils(840): end MOB compaction of files for table='testGetAfterCompaction', column='f1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-05 19:55:14,651 DEBUG [PEWorker-2] locking.LockProcedure(242): UNLOCKED pid=74, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE 2023-03-05 19:55:14,652 INFO [RS:0;jenkins-hbase4:37037-shortCompactions-0] throttle.PressureAwareThroughputController(145): 2f859dd19bfc7b5149ac61d5410de746#f1#compaction#76 average throughput is 0.08 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-05 19:55:14,652 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=74, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE in 879 msec 2023-03-05 19:55:14,687 INFO [Listener at localhost/40787] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/f1/387dcc6a6b6a4931bad7bcf5ecd7ac99; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678046113762; access_time=1678046112947; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-05 19:55:14,687 INFO [Listener at localhost/40787] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/f1/71b00fd6c6a9400bb51d628286b49cac_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678046114596; access_time=1678046113789; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-05 19:55:14,687 INFO [Listener at localhost/40787] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/f1/a84cae38d2f449e7aca1dd304e2c3d77; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678046112933; access_time=1678046112119; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-05 19:55:14,691 DEBUG [Listener at localhost/40787] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-03-05 19:55:14,694 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:37696, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-03-05 19:55:14,694 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37037] regionserver.RSRpcServices(1662): Compacting testGetAfterCompaction,,1678046109999.2f859dd19bfc7b5149ac61d5410de746. 2023-03-05 19:55:14,695 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37037] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 2 compacting, 0 eligible, 16 blocking 2023-03-05 19:55:14,695 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37037] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 0 files of size 0 starting at candidate #-1 after considering 0 permutations with 0 in ratio 2023-03-05 19:55:14,695 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37037] compactions.SortedCompactionPolicy(232): Not compacting files because we only have 0 files ready for compaction. Need 3 to initiate. 2023-03-05 19:55:14,695 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37037] regionserver.CompactSplit(415): Not compacting testGetAfterCompaction,,1678046109999.2f859dd19bfc7b5149ac61d5410de746. because compaction request was cancelled 2023-03-05 19:55:14,746 INFO [Listener at localhost/40787] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/f1/387dcc6a6b6a4931bad7bcf5ecd7ac99; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678046113762; access_time=1678046112947; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-05 19:55:14,747 INFO [Listener at localhost/40787] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/f1/71b00fd6c6a9400bb51d628286b49cac_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678046114596; access_time=1678046113789; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-05 19:55:14,747 INFO [Listener at localhost/40787] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/f1/a84cae38d2f449e7aca1dd304e2c3d77; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678046112933; access_time=1678046112119; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-05 19:55:14,798 INFO [Listener at localhost/40787] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/f1/387dcc6a6b6a4931bad7bcf5ecd7ac99; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678046113762; access_time=1678046112947; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-05 19:55:14,798 INFO [Listener at localhost/40787] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/f1/71b00fd6c6a9400bb51d628286b49cac_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678046114596; access_time=1678046113789; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-05 19:55:14,798 INFO [Listener at localhost/40787] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/f1/a84cae38d2f449e7aca1dd304e2c3d77; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678046112933; access_time=1678046112119; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-05 19:55:14,848 INFO [Listener at localhost/40787] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/f1/387dcc6a6b6a4931bad7bcf5ecd7ac99; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678046113762; access_time=1678046112947; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-05 19:55:14,849 INFO [Listener at localhost/40787] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/f1/71b00fd6c6a9400bb51d628286b49cac_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678046114596; access_time=1678046113789; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-05 19:55:14,849 INFO [Listener at localhost/40787] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/f1/a84cae38d2f449e7aca1dd304e2c3d77; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678046112933; access_time=1678046112119; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-05 19:55:14,899 INFO [Listener at localhost/40787] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/f1/387dcc6a6b6a4931bad7bcf5ecd7ac99; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678046113762; access_time=1678046112947; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-05 19:55:14,900 INFO [Listener at localhost/40787] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/f1/71b00fd6c6a9400bb51d628286b49cac_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678046114596; access_time=1678046113789; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-05 19:55:14,900 INFO [Listener at localhost/40787] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/f1/a84cae38d2f449e7aca1dd304e2c3d77; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678046112933; access_time=1678046112119; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-05 19:55:14,950 INFO [Listener at localhost/40787] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/f1/387dcc6a6b6a4931bad7bcf5ecd7ac99; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678046113762; access_time=1678046112947; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-05 19:55:14,951 INFO [Listener at localhost/40787] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/f1/71b00fd6c6a9400bb51d628286b49cac_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678046114596; access_time=1678046113789; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-05 19:55:14,951 INFO [Listener at localhost/40787] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/f1/a84cae38d2f449e7aca1dd304e2c3d77; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678046112933; access_time=1678046112119; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-05 19:55:15,001 INFO [Listener at localhost/40787] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/f1/387dcc6a6b6a4931bad7bcf5ecd7ac99; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678046113762; access_time=1678046112947; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-05 19:55:15,002 INFO [Listener at localhost/40787] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/f1/71b00fd6c6a9400bb51d628286b49cac_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678046114596; access_time=1678046113789; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-05 19:55:15,002 INFO [Listener at localhost/40787] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/f1/a84cae38d2f449e7aca1dd304e2c3d77; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678046112933; access_time=1678046112119; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-05 19:55:15,052 INFO [Listener at localhost/40787] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/f1/387dcc6a6b6a4931bad7bcf5ecd7ac99; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678046113762; access_time=1678046112947; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-05 19:55:15,052 INFO [Listener at localhost/40787] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/f1/71b00fd6c6a9400bb51d628286b49cac_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678046114596; access_time=1678046113789; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-05 19:55:15,053 INFO [Listener at localhost/40787] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/f1/a84cae38d2f449e7aca1dd304e2c3d77; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678046112933; access_time=1678046112119; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-05 19:55:15,103 INFO [Listener at localhost/40787] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/f1/387dcc6a6b6a4931bad7bcf5ecd7ac99; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678046113762; access_time=1678046112947; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-05 19:55:15,103 INFO [Listener at localhost/40787] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/f1/71b00fd6c6a9400bb51d628286b49cac_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678046114596; access_time=1678046113789; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-05 19:55:15,103 INFO [Listener at localhost/40787] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/f1/a84cae38d2f449e7aca1dd304e2c3d77; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678046112933; access_time=1678046112119; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-05 19:55:15,154 INFO [Listener at localhost/40787] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/f1/387dcc6a6b6a4931bad7bcf5ecd7ac99; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678046113762; access_time=1678046112947; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-05 19:55:15,154 INFO [Listener at localhost/40787] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/f1/71b00fd6c6a9400bb51d628286b49cac_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678046114596; access_time=1678046113789; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-05 19:55:15,154 INFO [Listener at localhost/40787] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/f1/a84cae38d2f449e7aca1dd304e2c3d77; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678046112933; access_time=1678046112119; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-05 19:55:15,205 INFO [Listener at localhost/40787] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/f1/387dcc6a6b6a4931bad7bcf5ecd7ac99; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678046113762; access_time=1678046112947; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-05 19:55:15,205 INFO [Listener at localhost/40787] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/f1/71b00fd6c6a9400bb51d628286b49cac_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678046114596; access_time=1678046113789; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-05 19:55:15,205 INFO [Listener at localhost/40787] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/f1/a84cae38d2f449e7aca1dd304e2c3d77; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678046112933; access_time=1678046112119; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-05 19:55:15,256 INFO [Listener at localhost/40787] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/f1/387dcc6a6b6a4931bad7bcf5ecd7ac99; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678046113762; access_time=1678046112947; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-05 19:55:15,256 INFO [Listener at localhost/40787] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/f1/71b00fd6c6a9400bb51d628286b49cac_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678046114596; access_time=1678046113789; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-05 19:55:15,256 INFO [Listener at localhost/40787] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/f1/a84cae38d2f449e7aca1dd304e2c3d77; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678046112933; access_time=1678046112119; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-05 19:55:15,307 INFO [Listener at localhost/40787] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/f1/387dcc6a6b6a4931bad7bcf5ecd7ac99; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678046113762; access_time=1678046112947; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-05 19:55:15,307 INFO [Listener at localhost/40787] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/f1/71b00fd6c6a9400bb51d628286b49cac_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678046114596; access_time=1678046113789; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-05 19:55:15,307 INFO [Listener at localhost/40787] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/f1/a84cae38d2f449e7aca1dd304e2c3d77; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678046112933; access_time=1678046112119; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-05 19:55:15,358 INFO [Listener at localhost/40787] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/f1/387dcc6a6b6a4931bad7bcf5ecd7ac99; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678046113762; access_time=1678046112947; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-05 19:55:15,358 INFO [Listener at localhost/40787] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/f1/71b00fd6c6a9400bb51d628286b49cac_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678046114596; access_time=1678046113789; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-05 19:55:15,358 INFO [Listener at localhost/40787] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/f1/a84cae38d2f449e7aca1dd304e2c3d77; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678046112933; access_time=1678046112119; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-05 19:55:15,409 INFO [Listener at localhost/40787] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/f1/387dcc6a6b6a4931bad7bcf5ecd7ac99; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678046113762; access_time=1678046112947; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-05 19:55:15,409 INFO [Listener at localhost/40787] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/f1/71b00fd6c6a9400bb51d628286b49cac_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678046114596; access_time=1678046113789; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-05 19:55:15,409 INFO [Listener at localhost/40787] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/f1/a84cae38d2f449e7aca1dd304e2c3d77; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678046112933; access_time=1678046112119; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-05 19:55:15,459 INFO [Listener at localhost/40787] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/f1/387dcc6a6b6a4931bad7bcf5ecd7ac99; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678046113762; access_time=1678046112947; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-05 19:55:15,459 INFO [Listener at localhost/40787] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/f1/71b00fd6c6a9400bb51d628286b49cac_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678046114596; access_time=1678046113789; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-05 19:55:15,460 INFO [Listener at localhost/40787] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/f1/a84cae38d2f449e7aca1dd304e2c3d77; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678046112933; access_time=1678046112119; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-05 19:55:15,468 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/.tmp/f1/7e8139e322b84bf8a7990ec1dabcf4a8 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/f1/7e8139e322b84bf8a7990ec1dabcf4a8 2023-03-05 19:55:15,473 INFO [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 2 file(s) in 2f859dd19bfc7b5149ac61d5410de746/f1 of 2f859dd19bfc7b5149ac61d5410de746 into 7e8139e322b84bf8a7990ec1dabcf4a8(size=5.1 K), total size for store is 10.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-05 19:55:15,473 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 2f859dd19bfc7b5149ac61d5410de746: 2023-03-05 19:55:15,473 INFO [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testGetAfterCompaction,,1678046109999.2f859dd19bfc7b5149ac61d5410de746., storeName=f1, fileCount=2, fileSize=10.0 K (4.9 K, 5.1 K), priority=2, time=1678046114643; duration=0sec 2023-03-05 19:55:15,473 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-05 19:55:15,497 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/f1/387dcc6a6b6a4931bad7bcf5ecd7ac99, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/f1/71b00fd6c6a9400bb51d628286b49cac_SeqId_12_] to archive 2023-03-05 19:55:15,498 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-05 19:55:15,499 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/f1/387dcc6a6b6a4931bad7bcf5ecd7ac99 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/f1/387dcc6a6b6a4931bad7bcf5ecd7ac99 2023-03-05 19:55:15,500 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/f1/71b00fd6c6a9400bb51d628286b49cac_SeqId_12_ to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/f1/71b00fd6c6a9400bb51d628286b49cac_SeqId_12_ 2023-03-05 19:55:15,510 INFO [Listener at localhost/40787] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/f1/7e8139e322b84bf8a7990ec1dabcf4a8; isDirectory=false; length=5228; replication=1; blocksize=134217728; modification_time=1678046115464; access_time=1678046114649; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-05 19:55:15,510 INFO [Listener at localhost/40787] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/f1/a84cae38d2f449e7aca1dd304e2c3d77; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678046112933; access_time=1678046112119; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-05 19:55:15,624 INFO [Listener at localhost/40787] client.HBaseAdmin$15(890): Started disable of testGetAfterCompaction 2023-03-05 19:55:15,628 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testGetAfterCompaction 2023-03-05 19:55:15,634 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure2.ProcedureExecutor(1029): Stored pid=75, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testGetAfterCompaction 2023-03-05 19:55:15,636 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678046115636"}]},"ts":"1678046115636"} 2023-03-05 19:55:15,637 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testGetAfterCompaction, state=DISABLING in hbase:meta 2023-03-05 19:55:15,638 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1227): Checking to see if procedure is done pid=75 2023-03-05 19:55:15,639 INFO [PEWorker-3] procedure.DisableTableProcedure(293): Set testGetAfterCompaction to state=DISABLING 2023-03-05 19:55:15,641 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=76, ppid=75, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testGetAfterCompaction, region=2f859dd19bfc7b5149ac61d5410de746, UNASSIGN}] 2023-03-05 19:55:15,642 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=76, ppid=75, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testGetAfterCompaction, region=2f859dd19bfc7b5149ac61d5410de746, UNASSIGN 2023-03-05 19:55:15,642 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=2f859dd19bfc7b5149ac61d5410de746, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:15,642 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testGetAfterCompaction,,1678046109999.2f859dd19bfc7b5149ac61d5410de746.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1678046115642"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678046115642"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678046115642"}]},"ts":"1678046115642"} 2023-03-05 19:55:15,643 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=77, ppid=76, state=RUNNABLE; CloseRegionProcedure 2f859dd19bfc7b5149ac61d5410de746, server=jenkins-hbase4.apache.org,37037,1678046018983}] 2023-03-05 19:55:15,738 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1227): Checking to see if procedure is done pid=75 2023-03-05 19:55:15,795 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 2f859dd19bfc7b5149ac61d5410de746 2023-03-05 19:55:15,795 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 2f859dd19bfc7b5149ac61d5410de746, disabling compactions & flushes 2023-03-05 19:55:15,795 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testGetAfterCompaction,,1678046109999.2f859dd19bfc7b5149ac61d5410de746. 2023-03-05 19:55:15,795 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testGetAfterCompaction,,1678046109999.2f859dd19bfc7b5149ac61d5410de746. 2023-03-05 19:55:15,795 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testGetAfterCompaction,,1678046109999.2f859dd19bfc7b5149ac61d5410de746. after waiting 0 ms 2023-03-05 19:55:15,796 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testGetAfterCompaction,,1678046109999.2f859dd19bfc7b5149ac61d5410de746. 2023-03-05 19:55:15,799 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/recovered.edits/15.seqid, newMaxSeqId=15, maxSeqId=1 2023-03-05 19:55:15,799 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro 2023-03-05 19:55:15,801 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testGetAfterCompaction,,1678046109999.2f859dd19bfc7b5149ac61d5410de746. 2023-03-05 19:55:15,801 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 2f859dd19bfc7b5149ac61d5410de746: 2023-03-05 19:55:15,802 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 2f859dd19bfc7b5149ac61d5410de746 2023-03-05 19:55:15,802 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=2f859dd19bfc7b5149ac61d5410de746, regionState=CLOSED 2023-03-05 19:55:15,802 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testGetAfterCompaction,,1678046109999.2f859dd19bfc7b5149ac61d5410de746.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1678046115802"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678046115802"}]},"ts":"1678046115802"} 2023-03-05 19:55:15,805 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=77, resume processing ppid=76 2023-03-05 19:55:15,805 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=77, ppid=76, state=SUCCESS; CloseRegionProcedure 2f859dd19bfc7b5149ac61d5410de746, server=jenkins-hbase4.apache.org,37037,1678046018983 in 160 msec 2023-03-05 19:55:15,806 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=76, resume processing ppid=75 2023-03-05 19:55:15,806 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=76, ppid=75, state=SUCCESS; TransitRegionStateProcedure table=testGetAfterCompaction, region=2f859dd19bfc7b5149ac61d5410de746, UNASSIGN in 164 msec 2023-03-05 19:55:15,806 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678046115806"}]},"ts":"1678046115806"} 2023-03-05 19:55:15,807 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testGetAfterCompaction, state=DISABLED in hbase:meta 2023-03-05 19:55:15,813 INFO [PEWorker-3] procedure.DisableTableProcedure(305): Set testGetAfterCompaction to state=DISABLED 2023-03-05 19:55:15,814 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=75, state=SUCCESS; DisableTableProcedure table=testGetAfterCompaction in 184 msec 2023-03-05 19:55:15,939 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1227): Checking to see if procedure is done pid=75 2023-03-05 19:55:15,940 INFO [Listener at localhost/40787] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testGetAfterCompaction, procId: 75 completed 2023-03-05 19:55:15,944 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete testGetAfterCompaction 2023-03-05 19:55:15,950 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure2.ProcedureExecutor(1029): Stored pid=78, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testGetAfterCompaction 2023-03-05 19:55:15,952 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=78, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-03-05 19:55:15,954 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=78, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-03-05 19:55:15,955 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1227): Checking to see if procedure is done pid=78 2023-03-05 19:55:15,957 DEBUG [HFileArchiver-12] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/.tmp/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746 2023-03-05 19:55:15,959 DEBUG [HFileArchiver-12] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/.tmp/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/f1, FileablePath, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/.tmp/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/recovered.edits] 2023-03-05 19:55:15,962 DEBUG [HFileArchiver-12] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/.tmp/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/f1/7e8139e322b84bf8a7990ec1dabcf4a8 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/f1/7e8139e322b84bf8a7990ec1dabcf4a8 2023-03-05 19:55:15,963 DEBUG [HFileArchiver-12] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/.tmp/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/f1/a84cae38d2f449e7aca1dd304e2c3d77 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/f1/a84cae38d2f449e7aca1dd304e2c3d77 2023-03-05 19:55:15,965 DEBUG [HFileArchiver-12] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/.tmp/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/recovered.edits/15.seqid to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746/recovered.edits/15.seqid 2023-03-05 19:55:15,966 DEBUG [HFileArchiver-12] backup.HFileArchiver(596): Deleted hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/.tmp/data/default/testGetAfterCompaction/2f859dd19bfc7b5149ac61d5410de746 2023-03-05 19:55:15,966 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived testGetAfterCompaction regions 2023-03-05 19:55:15,966 DEBUG [PEWorker-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0 2023-03-05 19:55:15,967 DEBUG [PEWorker-4] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1] 2023-03-05 19:55:15,969 DEBUG [PEWorker-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e202303054c4d0b51776d4494b98abd70817af3e1 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e202303054c4d0b51776d4494b98abd70817af3e1 2023-03-05 19:55:15,970 DEBUG [PEWorker-4] backup.HFileArchiver(596): Deleted hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0 2023-03-05 19:55:15,972 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=78, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-03-05 19:55:15,992 WARN [PEWorker-4] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testGetAfterCompaction from hbase:meta 2023-03-05 19:55:15,994 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(421): Removing 'testGetAfterCompaction' descriptor. 2023-03-05 19:55:15,995 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=78, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-03-05 19:55:15,995 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(411): Removing 'testGetAfterCompaction' from region states. 2023-03-05 19:55:15,995 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testGetAfterCompaction,,1678046109999.2f859dd19bfc7b5149ac61d5410de746.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1678046115995"}]},"ts":"9223372036854775807"} 2023-03-05 19:55:15,997 INFO [PEWorker-4] hbase.MetaTableAccessor(1788): Deleted 1 regions from META 2023-03-05 19:55:15,997 DEBUG [PEWorker-4] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 2f859dd19bfc7b5149ac61d5410de746, NAME => 'testGetAfterCompaction,,1678046109999.2f859dd19bfc7b5149ac61d5410de746.', STARTKEY => '', ENDKEY => ''}] 2023-03-05 19:55:15,997 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(415): Marking 'testGetAfterCompaction' as deleted. 2023-03-05 19:55:15,997 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1678046115997"}]},"ts":"9223372036854775807"} 2023-03-05 19:55:15,998 INFO [PEWorker-4] hbase.MetaTableAccessor(1658): Deleted table testGetAfterCompaction state from META 2023-03-05 19:55:16,000 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(130): Finished pid=78, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-03-05 19:55:16,001 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=78, state=SUCCESS; DeleteTableProcedure table=testGetAfterCompaction in 55 msec 2023-03-05 19:55:16,056 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1227): Checking to see if procedure is done pid=78 2023-03-05 19:55:16,056 INFO [Listener at localhost/40787] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testGetAfterCompaction, procId: 78 completed 2023-03-05 19:55:16,070 INFO [Listener at localhost/40787] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testGetAfterCompaction Thread=244 (was 240) - Thread LEAK? -, OpenFileDescriptor=616 (was 620), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=140 (was 152), ProcessCount=173 (was 173), AvailableMemoryMB=4666 (was 4673) 2023-03-05 19:55:16,080 INFO [Listener at localhost/40787] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testCompactionWithHFileLink Thread=244, OpenFileDescriptor=616, MaxFileDescriptor=60000, SystemLoadAverage=140, ProcessCount=173, AvailableMemoryMB=4665 2023-03-05 19:55:16,082 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testCompactionWithHFileLink', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-03-05 19:55:16,083 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure2.ProcedureExecutor(1029): Stored pid=79, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testCompactionWithHFileLink 2023-03-05 19:55:16,084 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_PRE_OPERATION 2023-03-05 19:55:16,084 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testCompactionWithHFileLink" procId is: 79 2023-03-05 19:55:16,084 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-05 19:55:16,085 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-03-05 19:55:16,086 DEBUG [HFileArchiver-13] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/.tmp/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a 2023-03-05 19:55:16,086 DEBUG [HFileArchiver-14] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/.tmp/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d 2023-03-05 19:55:16,086 DEBUG [HFileArchiver-15] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/.tmp/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6 2023-03-05 19:55:16,086 DEBUG [HFileArchiver-13] backup.HFileArchiver(153): Directory hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/.tmp/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a empty. 2023-03-05 19:55:16,086 DEBUG [HFileArchiver-14] backup.HFileArchiver(153): Directory hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/.tmp/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d empty. 2023-03-05 19:55:16,086 DEBUG [HFileArchiver-15] backup.HFileArchiver(153): Directory hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/.tmp/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6 empty. 2023-03-05 19:55:16,087 DEBUG [HFileArchiver-14] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/.tmp/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d 2023-03-05 19:55:16,087 DEBUG [HFileArchiver-15] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/.tmp/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6 2023-03-05 19:55:16,087 DEBUG [HFileArchiver-13] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/.tmp/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a 2023-03-05 19:55:16,087 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testCompactionWithHFileLink regions 2023-03-05 19:55:16,185 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-03-05 19:55:16,386 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-03-05 19:55:16,495 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/.tmp/data/default/testCompactionWithHFileLink/.tabledesc/.tableinfo.0000000001 2023-03-05 19:55:16,496 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(7675): creating {ENCODED => 942afd2d0a4ceef025fd5a607956d53d, NAME => 'testCompactionWithHFileLink,1,1678046116082.942afd2d0a4ceef025fd5a607956d53d.', STARTKEY => '1', ENDKEY => '2'}, tableDescriptor='testCompactionWithHFileLink', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/.tmp 2023-03-05 19:55:16,497 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(7675): creating {ENCODED => daa57ebba7c140c319a2f6facc725e4a, NAME => 'testCompactionWithHFileLink,,1678046116082.daa57ebba7c140c319a2f6facc725e4a.', STARTKEY => '', ENDKEY => '1'}, tableDescriptor='testCompactionWithHFileLink', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/.tmp 2023-03-05 19:55:16,497 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(7675): creating {ENCODED => 25af79a83c9b248c956b57fc8507ead6, NAME => 'testCompactionWithHFileLink,2,1678046116082.25af79a83c9b248c956b57fc8507ead6.', STARTKEY => '2', ENDKEY => ''}, tableDescriptor='testCompactionWithHFileLink', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/.tmp 2023-03-05 19:55:16,503 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,,1678046116082.daa57ebba7c140c319a2f6facc725e4a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-05 19:55:16,503 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1604): Closing daa57ebba7c140c319a2f6facc725e4a, disabling compactions & flushes 2023-03-05 19:55:16,503 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,,1678046116082.daa57ebba7c140c319a2f6facc725e4a. 2023-03-05 19:55:16,503 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,,1678046116082.daa57ebba7c140c319a2f6facc725e4a. 2023-03-05 19:55:16,503 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,,1678046116082.daa57ebba7c140c319a2f6facc725e4a. after waiting 0 ms 2023-03-05 19:55:16,503 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,,1678046116082.daa57ebba7c140c319a2f6facc725e4a. 2023-03-05 19:55:16,503 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,,1678046116082.daa57ebba7c140c319a2f6facc725e4a. 2023-03-05 19:55:16,503 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1558): Region close journal for daa57ebba7c140c319a2f6facc725e4a: 2023-03-05 19:55:16,687 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-03-05 19:55:16,902 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,1,1678046116082.942afd2d0a4ceef025fd5a607956d53d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-05 19:55:16,902 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,2,1678046116082.25af79a83c9b248c956b57fc8507ead6.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-05 19:55:16,903 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1604): Closing 942afd2d0a4ceef025fd5a607956d53d, disabling compactions & flushes 2023-03-05 19:55:16,903 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1604): Closing 25af79a83c9b248c956b57fc8507ead6, disabling compactions & flushes 2023-03-05 19:55:16,903 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,1,1678046116082.942afd2d0a4ceef025fd5a607956d53d. 2023-03-05 19:55:16,903 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,2,1678046116082.25af79a83c9b248c956b57fc8507ead6. 2023-03-05 19:55:16,903 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,1,1678046116082.942afd2d0a4ceef025fd5a607956d53d. 2023-03-05 19:55:16,903 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,2,1678046116082.25af79a83c9b248c956b57fc8507ead6. 2023-03-05 19:55:16,903 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,1,1678046116082.942afd2d0a4ceef025fd5a607956d53d. after waiting 0 ms 2023-03-05 19:55:16,903 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,2,1678046116082.25af79a83c9b248c956b57fc8507ead6. after waiting 0 ms 2023-03-05 19:55:16,903 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,2,1678046116082.25af79a83c9b248c956b57fc8507ead6. 2023-03-05 19:55:16,903 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,1,1678046116082.942afd2d0a4ceef025fd5a607956d53d. 2023-03-05 19:55:16,903 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,2,1678046116082.25af79a83c9b248c956b57fc8507ead6. 2023-03-05 19:55:16,903 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,1,1678046116082.942afd2d0a4ceef025fd5a607956d53d. 2023-03-05 19:55:16,903 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1558): Region close journal for 25af79a83c9b248c956b57fc8507ead6: 2023-03-05 19:55:16,903 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1558): Region close journal for 942afd2d0a4ceef025fd5a607956d53d: 2023-03-05 19:55:16,905 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_ADD_TO_META 2023-03-05 19:55:16,906 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,,1678046116082.daa57ebba7c140c319a2f6facc725e4a.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678046116906"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678046116906"}]},"ts":"1678046116906"} 2023-03-05 19:55:16,906 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,2,1678046116082.25af79a83c9b248c956b57fc8507ead6.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678046116906"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678046116906"}]},"ts":"1678046116906"} 2023-03-05 19:55:16,906 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,1,1678046116082.942afd2d0a4ceef025fd5a607956d53d.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1678046116906"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678046116906"}]},"ts":"1678046116906"} 2023-03-05 19:55:16,908 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-03-05 19:55:16,908 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-05 19:55:16,908 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678046116908"}]},"ts":"1678046116908"} 2023-03-05 19:55:16,909 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=ENABLING in hbase:meta 2023-03-05 19:55:16,914 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=daa57ebba7c140c319a2f6facc725e4a, ASSIGN}, {pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=942afd2d0a4ceef025fd5a607956d53d, ASSIGN}, {pid=82, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=25af79a83c9b248c956b57fc8507ead6, ASSIGN}] 2023-03-05 19:55:16,915 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=942afd2d0a4ceef025fd5a607956d53d, ASSIGN 2023-03-05 19:55:16,915 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=daa57ebba7c140c319a2f6facc725e4a, ASSIGN 2023-03-05 19:55:16,915 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=82, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=25af79a83c9b248c956b57fc8507ead6, ASSIGN 2023-03-05 19:55:16,915 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=942afd2d0a4ceef025fd5a607956d53d, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,37037,1678046018983; forceNewPlan=false, retain=false 2023-03-05 19:55:16,915 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=daa57ebba7c140c319a2f6facc725e4a, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,37037,1678046018983; forceNewPlan=false, retain=false 2023-03-05 19:55:16,915 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=82, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=25af79a83c9b248c956b57fc8507ead6, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,37037,1678046018983; forceNewPlan=false, retain=false 2023-03-05 19:55:17,067 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=82 updating hbase:meta row=25af79a83c9b248c956b57fc8507ead6, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:17,067 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=942afd2d0a4ceef025fd5a607956d53d, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:17,067 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=daa57ebba7c140c319a2f6facc725e4a, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:17,067 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,2,1678046116082.25af79a83c9b248c956b57fc8507ead6.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678046117067"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678046117067"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678046117067"}]},"ts":"1678046117067"} 2023-03-05 19:55:17,067 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,1,1678046116082.942afd2d0a4ceef025fd5a607956d53d.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1678046117067"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678046117067"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678046117067"}]},"ts":"1678046117067"} 2023-03-05 19:55:17,067 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,,1678046116082.daa57ebba7c140c319a2f6facc725e4a.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678046117067"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678046117067"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678046117067"}]},"ts":"1678046117067"} 2023-03-05 19:55:17,069 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=83, ppid=82, state=RUNNABLE; OpenRegionProcedure 25af79a83c9b248c956b57fc8507ead6, server=jenkins-hbase4.apache.org,37037,1678046018983}] 2023-03-05 19:55:17,069 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=84, ppid=81, state=RUNNABLE; OpenRegionProcedure 942afd2d0a4ceef025fd5a607956d53d, server=jenkins-hbase4.apache.org,37037,1678046018983}] 2023-03-05 19:55:17,071 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=85, ppid=80, state=RUNNABLE; OpenRegionProcedure daa57ebba7c140c319a2f6facc725e4a, server=jenkins-hbase4.apache.org,37037,1678046018983}] 2023-03-05 19:55:17,188 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-03-05 19:55:17,223 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,,1678046116082.daa57ebba7c140c319a2f6facc725e4a. 2023-03-05 19:55:17,224 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => daa57ebba7c140c319a2f6facc725e4a, NAME => 'testCompactionWithHFileLink,,1678046116082.daa57ebba7c140c319a2f6facc725e4a.', STARTKEY => '', ENDKEY => '1'} 2023-03-05 19:55:17,224 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink daa57ebba7c140c319a2f6facc725e4a 2023-03-05 19:55:17,224 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,,1678046116082.daa57ebba7c140c319a2f6facc725e4a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-05 19:55:17,224 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for daa57ebba7c140c319a2f6facc725e4a 2023-03-05 19:55:17,224 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for daa57ebba7c140c319a2f6facc725e4a 2023-03-05 19:55:17,225 INFO [StoreOpener-daa57ebba7c140c319a2f6facc725e4a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region daa57ebba7c140c319a2f6facc725e4a 2023-03-05 19:55:17,226 DEBUG [StoreOpener-daa57ebba7c140c319a2f6facc725e4a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family1 2023-03-05 19:55:17,226 DEBUG [StoreOpener-daa57ebba7c140c319a2f6facc725e4a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family1 2023-03-05 19:55:17,227 INFO [StoreOpener-daa57ebba7c140c319a2f6facc725e4a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region daa57ebba7c140c319a2f6facc725e4a columnFamilyName family1 2023-03-05 19:55:17,228 INFO [StoreOpener-daa57ebba7c140c319a2f6facc725e4a-1] regionserver.HStore(310): Store=daa57ebba7c140c319a2f6facc725e4a/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-05 19:55:17,228 INFO [StoreOpener-daa57ebba7c140c319a2f6facc725e4a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region daa57ebba7c140c319a2f6facc725e4a 2023-03-05 19:55:17,229 DEBUG [StoreOpener-daa57ebba7c140c319a2f6facc725e4a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family2 2023-03-05 19:55:17,229 DEBUG [StoreOpener-daa57ebba7c140c319a2f6facc725e4a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family2 2023-03-05 19:55:17,230 INFO [StoreOpener-daa57ebba7c140c319a2f6facc725e4a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region daa57ebba7c140c319a2f6facc725e4a columnFamilyName family2 2023-03-05 19:55:17,231 INFO [StoreOpener-daa57ebba7c140c319a2f6facc725e4a-1] regionserver.HStore(310): Store=daa57ebba7c140c319a2f6facc725e4a/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-05 19:55:17,231 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a 2023-03-05 19:55:17,232 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a 2023-03-05 19:55:17,233 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-05 19:55:17,234 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for daa57ebba7c140c319a2f6facc725e4a 2023-03-05 19:55:17,236 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-05 19:55:17,236 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened daa57ebba7c140c319a2f6facc725e4a; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63054035, jitterRate=-0.0604216605424881}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-05 19:55:17,236 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for daa57ebba7c140c319a2f6facc725e4a: 2023-03-05 19:55:17,237 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,,1678046116082.daa57ebba7c140c319a2f6facc725e4a., pid=85, masterSystemTime=1678046117221 2023-03-05 19:55:17,238 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,,1678046116082.daa57ebba7c140c319a2f6facc725e4a. 2023-03-05 19:55:17,238 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,,1678046116082.daa57ebba7c140c319a2f6facc725e4a. 2023-03-05 19:55:17,238 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,2,1678046116082.25af79a83c9b248c956b57fc8507ead6. 2023-03-05 19:55:17,238 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 25af79a83c9b248c956b57fc8507ead6, NAME => 'testCompactionWithHFileLink,2,1678046116082.25af79a83c9b248c956b57fc8507ead6.', STARTKEY => '2', ENDKEY => ''} 2023-03-05 19:55:17,238 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink 25af79a83c9b248c956b57fc8507ead6 2023-03-05 19:55:17,238 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=daa57ebba7c140c319a2f6facc725e4a, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:17,238 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,2,1678046116082.25af79a83c9b248c956b57fc8507ead6.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-05 19:55:17,239 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 25af79a83c9b248c956b57fc8507ead6 2023-03-05 19:55:17,239 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 25af79a83c9b248c956b57fc8507ead6 2023-03-05 19:55:17,239 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,,1678046116082.daa57ebba7c140c319a2f6facc725e4a.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678046117238"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678046117238"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678046117238"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678046117238"}]},"ts":"1678046117238"} 2023-03-05 19:55:17,240 INFO [StoreOpener-25af79a83c9b248c956b57fc8507ead6-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 25af79a83c9b248c956b57fc8507ead6 2023-03-05 19:55:17,241 DEBUG [StoreOpener-25af79a83c9b248c956b57fc8507ead6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family1 2023-03-05 19:55:17,241 DEBUG [StoreOpener-25af79a83c9b248c956b57fc8507ead6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family1 2023-03-05 19:55:17,241 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=85, resume processing ppid=80 2023-03-05 19:55:17,241 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=85, ppid=80, state=SUCCESS; OpenRegionProcedure daa57ebba7c140c319a2f6facc725e4a, server=jenkins-hbase4.apache.org,37037,1678046018983 in 170 msec 2023-03-05 19:55:17,241 INFO [StoreOpener-25af79a83c9b248c956b57fc8507ead6-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 25af79a83c9b248c956b57fc8507ead6 columnFamilyName family1 2023-03-05 19:55:17,242 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=80, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=daa57ebba7c140c319a2f6facc725e4a, ASSIGN in 329 msec 2023-03-05 19:55:17,242 INFO [StoreOpener-25af79a83c9b248c956b57fc8507ead6-1] regionserver.HStore(310): Store=25af79a83c9b248c956b57fc8507ead6/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-05 19:55:17,243 INFO [StoreOpener-25af79a83c9b248c956b57fc8507ead6-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 25af79a83c9b248c956b57fc8507ead6 2023-03-05 19:55:17,244 DEBUG [StoreOpener-25af79a83c9b248c956b57fc8507ead6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family2 2023-03-05 19:55:17,244 DEBUG [StoreOpener-25af79a83c9b248c956b57fc8507ead6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family2 2023-03-05 19:55:17,244 INFO [StoreOpener-25af79a83c9b248c956b57fc8507ead6-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 25af79a83c9b248c956b57fc8507ead6 columnFamilyName family2 2023-03-05 19:55:17,245 INFO [StoreOpener-25af79a83c9b248c956b57fc8507ead6-1] regionserver.HStore(310): Store=25af79a83c9b248c956b57fc8507ead6/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-05 19:55:17,246 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6 2023-03-05 19:55:17,246 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6 2023-03-05 19:55:17,247 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-05 19:55:17,248 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 25af79a83c9b248c956b57fc8507ead6 2023-03-05 19:55:17,250 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-05 19:55:17,250 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 25af79a83c9b248c956b57fc8507ead6; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61770806, jitterRate=-0.07954326272010803}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-05 19:55:17,250 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 25af79a83c9b248c956b57fc8507ead6: 2023-03-05 19:55:17,251 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,2,1678046116082.25af79a83c9b248c956b57fc8507ead6., pid=83, masterSystemTime=1678046117221 2023-03-05 19:55:17,252 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,2,1678046116082.25af79a83c9b248c956b57fc8507ead6. 2023-03-05 19:55:17,252 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,2,1678046116082.25af79a83c9b248c956b57fc8507ead6. 2023-03-05 19:55:17,252 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,1,1678046116082.942afd2d0a4ceef025fd5a607956d53d. 2023-03-05 19:55:17,252 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 942afd2d0a4ceef025fd5a607956d53d, NAME => 'testCompactionWithHFileLink,1,1678046116082.942afd2d0a4ceef025fd5a607956d53d.', STARTKEY => '1', ENDKEY => '2'} 2023-03-05 19:55:17,252 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=82 updating hbase:meta row=25af79a83c9b248c956b57fc8507ead6, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:17,252 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink 942afd2d0a4ceef025fd5a607956d53d 2023-03-05 19:55:17,252 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,2,1678046116082.25af79a83c9b248c956b57fc8507ead6.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678046117252"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678046117252"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678046117252"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678046117252"}]},"ts":"1678046117252"} 2023-03-05 19:55:17,252 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,1,1678046116082.942afd2d0a4ceef025fd5a607956d53d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-05 19:55:17,253 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 942afd2d0a4ceef025fd5a607956d53d 2023-03-05 19:55:17,253 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 942afd2d0a4ceef025fd5a607956d53d 2023-03-05 19:55:17,254 INFO [StoreOpener-942afd2d0a4ceef025fd5a607956d53d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 942afd2d0a4ceef025fd5a607956d53d 2023-03-05 19:55:17,255 DEBUG [StoreOpener-942afd2d0a4ceef025fd5a607956d53d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family1 2023-03-05 19:55:17,255 DEBUG [StoreOpener-942afd2d0a4ceef025fd5a607956d53d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family1 2023-03-05 19:55:17,255 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=83, resume processing ppid=82 2023-03-05 19:55:17,255 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=83, ppid=82, state=SUCCESS; OpenRegionProcedure 25af79a83c9b248c956b57fc8507ead6, server=jenkins-hbase4.apache.org,37037,1678046018983 in 185 msec 2023-03-05 19:55:17,255 INFO [StoreOpener-942afd2d0a4ceef025fd5a607956d53d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 942afd2d0a4ceef025fd5a607956d53d columnFamilyName family1 2023-03-05 19:55:17,256 INFO [StoreOpener-942afd2d0a4ceef025fd5a607956d53d-1] regionserver.HStore(310): Store=942afd2d0a4ceef025fd5a607956d53d/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-05 19:55:17,256 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=82, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=25af79a83c9b248c956b57fc8507ead6, ASSIGN in 342 msec 2023-03-05 19:55:17,256 INFO [StoreOpener-942afd2d0a4ceef025fd5a607956d53d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 942afd2d0a4ceef025fd5a607956d53d 2023-03-05 19:55:17,257 DEBUG [StoreOpener-942afd2d0a4ceef025fd5a607956d53d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family2 2023-03-05 19:55:17,257 DEBUG [StoreOpener-942afd2d0a4ceef025fd5a607956d53d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family2 2023-03-05 19:55:17,258 INFO [StoreOpener-942afd2d0a4ceef025fd5a607956d53d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 942afd2d0a4ceef025fd5a607956d53d columnFamilyName family2 2023-03-05 19:55:17,258 INFO [StoreOpener-942afd2d0a4ceef025fd5a607956d53d-1] regionserver.HStore(310): Store=942afd2d0a4ceef025fd5a607956d53d/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-05 19:55:17,259 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d 2023-03-05 19:55:17,259 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d 2023-03-05 19:55:17,261 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-05 19:55:17,262 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 942afd2d0a4ceef025fd5a607956d53d 2023-03-05 19:55:17,263 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-05 19:55:17,263 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 942afd2d0a4ceef025fd5a607956d53d; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64016888, jitterRate=-0.0460740327835083}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-05 19:55:17,264 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 942afd2d0a4ceef025fd5a607956d53d: 2023-03-05 19:55:17,264 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,1,1678046116082.942afd2d0a4ceef025fd5a607956d53d., pid=84, masterSystemTime=1678046117221 2023-03-05 19:55:17,265 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,1,1678046116082.942afd2d0a4ceef025fd5a607956d53d. 2023-03-05 19:55:17,265 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,1,1678046116082.942afd2d0a4ceef025fd5a607956d53d. 2023-03-05 19:55:17,266 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=942afd2d0a4ceef025fd5a607956d53d, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:17,266 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,1,1678046116082.942afd2d0a4ceef025fd5a607956d53d.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1678046117265"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678046117265"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678046117265"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678046117265"}]},"ts":"1678046117265"} 2023-03-05 19:55:17,268 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=84, resume processing ppid=81 2023-03-05 19:55:17,268 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=84, ppid=81, state=SUCCESS; OpenRegionProcedure 942afd2d0a4ceef025fd5a607956d53d, server=jenkins-hbase4.apache.org,37037,1678046018983 in 198 msec 2023-03-05 19:55:17,269 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=81, resume processing ppid=79 2023-03-05 19:55:17,269 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=81, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=942afd2d0a4ceef025fd5a607956d53d, ASSIGN in 356 msec 2023-03-05 19:55:17,270 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-05 19:55:17,270 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678046117270"}]},"ts":"1678046117270"} 2023-03-05 19:55:17,271 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=ENABLED in hbase:meta 2023-03-05 19:55:17,274 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_POST_OPERATION 2023-03-05 19:55:17,275 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=79, state=SUCCESS; CreateTableProcedure table=testCompactionWithHFileLink in 1.1920 sec 2023-03-05 19:55:18,189 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-03-05 19:55:18,189 INFO [Listener at localhost/40787] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testCompactionWithHFileLink, procId: 79 completed 2023-03-05 19:55:18,195 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,,1678046116082.daa57ebba7c140c319a2f6facc725e4a. with WAL disabled. Data may be lost in the event of a crash. 2023-03-05 19:55:18,197 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,1,1678046116082.942afd2d0a4ceef025fd5a607956d53d. with WAL disabled. Data may be lost in the event of a crash. 2023-03-05 19:55:18,198 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,2,1678046116082.25af79a83c9b248c956b57fc8507ead6. with WAL disabled. Data may be lost in the event of a crash. 2023-03-05 19:55:18,200 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-05 19:55:18,202 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-03-05 19:55:18,202 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-03-05 19:55:18,202 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:55:18,202 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-03-05 19:55:18,202 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-03-05 19:55:18,202 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-05 19:55:18,202 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-05 19:55:18,204 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:18,204 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:55:18,204 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:55:18,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:55:18,204 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:18,204 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-05 19:55:18,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-05 19:55:18,204 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-05 19:55:18,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-05 19:55:18,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-05 19:55:18,204 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-03-05 19:55:18,205 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-03-05 19:55:18,205 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-03-05 19:55:18,205 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:55:18,205 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-03-05 19:55:18,205 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1678046116082.942afd2d0a4ceef025fd5a607956d53d. 2023-03-05 19:55:18,205 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-05 19:55:18,205 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1678046116082.25af79a83c9b248c956b57fc8507ead6. 2023-03-05 19:55:18,205 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1678046116082.daa57ebba7c140c319a2f6facc725e4a. 2023-03-05 19:55:18,205 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1678046116082.25af79a83c9b248c956b57fc8507ead6. started... 2023-03-05 19:55:18,205 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-05 19:55:18,205 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1678046116082.942afd2d0a4ceef025fd5a607956d53d. started... 2023-03-05 19:55:18,205 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1678046116082.daa57ebba7c140c319a2f6facc725e4a. started... 2023-03-05 19:55:18,206 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 25af79a83c9b248c956b57fc8507ead6 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-03-05 19:55:18,206 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 942afd2d0a4ceef025fd5a607956d53d 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-03-05 19:55:18,206 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing daa57ebba7c140c319a2f6facc725e4a 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-03-05 19:55:18,225 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023030524196f3846324173b9e8c56ba282d604 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023030524196f3846324173b9e8c56ba282d604 2023-03-05 19:55:18,622 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230305202e97351c394119851f4075ef789695 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230305202e97351c394119851f4075ef789695 2023-03-05 19:55:18,625 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230305a33235cee5224842811abef82371aea6 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230305a33235cee5224842811abef82371aea6 2023-03-05 19:55:18,629 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/.tmp/family1/9a9155a47d074f5f9005ddf6337e173e 2023-03-05 19:55:19,027 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/.tmp/family1/2ddb935fe9dc46778a4c450ff10d3c44 2023-03-05 19:55:19,030 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/.tmp/family1/cdcb9dd831794696af2aaf30754840ee 2023-03-05 19:55:19,041 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230305f7d80ee747a04a6ca514f2b4ec56d1a4 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230305f7d80ee747a04a6ca514f2b4ec56d1a4 2023-03-05 19:55:19,045 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/.tmp/family2/22cbfc0fa2af442f976458cd740e398d 2023-03-05 19:55:19,048 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/.tmp/family1/9a9155a47d074f5f9005ddf6337e173e as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family1/9a9155a47d074f5f9005ddf6337e173e 2023-03-05 19:55:19,051 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family1/9a9155a47d074f5f9005ddf6337e173e, entries=4, sequenceid=5, filesize=5.4 K 2023-03-05 19:55:19,051 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/.tmp/family2/22cbfc0fa2af442f976458cd740e398d as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family2/22cbfc0fa2af442f976458cd740e398d 2023-03-05 19:55:19,054 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family2/22cbfc0fa2af442f976458cd740e398d, entries=2, sequenceid=5, filesize=5.1 K 2023-03-05 19:55:19,055 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 25af79a83c9b248c956b57fc8507ead6 in 849ms, sequenceid=5, compaction requested=false 2023-03-05 19:55:19,055 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testCompactionWithHFileLink' 2023-03-05 19:55:19,055 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 25af79a83c9b248c956b57fc8507ead6: 2023-03-05 19:55:19,055 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1678046116082.25af79a83c9b248c956b57fc8507ead6. 2023-03-05 19:55:19,055 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-05 19:55:19,441 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303052c52d973fc3d4c1d97d47e99d0c89ee5 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202303052c52d973fc3d4c1d97d47e99d0c89ee5 2023-03-05 19:55:19,442 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023030553b3fcde254241d4b99a405451edd672 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023030553b3fcde254241d4b99a405451edd672 2023-03-05 19:55:19,446 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/.tmp/family2/c42309b45f3243f2a021144945ad30f4 2023-03-05 19:55:19,450 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/.tmp/family1/cdcb9dd831794696af2aaf30754840ee as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family1/cdcb9dd831794696af2aaf30754840ee 2023-03-05 19:55:19,453 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family1/cdcb9dd831794696af2aaf30754840ee, entries=4, sequenceid=5, filesize=5.4 K 2023-03-05 19:55:19,453 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/.tmp/family2/c42309b45f3243f2a021144945ad30f4 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family2/c42309b45f3243f2a021144945ad30f4 2023-03-05 19:55:19,456 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family2/c42309b45f3243f2a021144945ad30f4, entries=2, sequenceid=5, filesize=5.1 K 2023-03-05 19:55:19,457 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for daa57ebba7c140c319a2f6facc725e4a in 1251ms, sequenceid=5, compaction requested=false 2023-03-05 19:55:19,458 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for daa57ebba7c140c319a2f6facc725e4a: 2023-03-05 19:55:19,458 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1678046116082.daa57ebba7c140c319a2f6facc725e4a. 2023-03-05 19:55:19,458 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-05 19:55:19,847 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/.tmp/family2/9a13755a01ce4e01b7e321e95b43b8cc 2023-03-05 19:55:19,850 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/.tmp/family1/2ddb935fe9dc46778a4c450ff10d3c44 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family1/2ddb935fe9dc46778a4c450ff10d3c44 2023-03-05 19:55:19,853 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family1/2ddb935fe9dc46778a4c450ff10d3c44, entries=4, sequenceid=5, filesize=5.4 K 2023-03-05 19:55:19,854 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/.tmp/family2/9a13755a01ce4e01b7e321e95b43b8cc as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family2/9a13755a01ce4e01b7e321e95b43b8cc 2023-03-05 19:55:19,857 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family2/9a13755a01ce4e01b7e321e95b43b8cc, entries=2, sequenceid=5, filesize=5.1 K 2023-03-05 19:55:19,858 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 942afd2d0a4ceef025fd5a607956d53d in 1652ms, sequenceid=5, compaction requested=false 2023-03-05 19:55:19,858 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 942afd2d0a4ceef025fd5a607956d53d: 2023-03-05 19:55:19,858 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1678046116082.942afd2d0a4ceef025fd5a607956d53d. 2023-03-05 19:55:19,858 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-05 19:55:19,858 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-05 19:55:19,858 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:19,858 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-03-05 19:55:19,858 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-03-05 19:55:19,860 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-05 19:55:19,860 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:19,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:19,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:55:19,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:55:19,860 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-05 19:55:19,860 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-05 19:55:19,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:55:19,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:55:19,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-05 19:55:19,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:19,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:55:19,861 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-03-05 19:55:19,861 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@18dce162[Count = 0] remaining members to acquire global barrier 2023-03-05 19:55:19,861 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-03-05 19:55:19,861 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-05 19:55:19,862 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-05 19:55:19,862 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-05 19:55:19,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-05 19:55:19,862 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-03-05 19:55:19,862 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-03-05 19:55:19,862 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,37037,1678046018983' in zk 2023-03-05 19:55:19,862 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:19,863 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-05 19:55:19,865 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:19,865 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-03-05 19:55:19,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:19,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:55:19,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:55:19,865 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:55:19,865 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-03-05 19:55:19,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:55:19,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:55:19,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-05 19:55:19,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:19,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:55:19,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-05 19:55:19,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:19,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,37037,1678046018983': 2023-03-05 19:55:19,866 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-03-05 19:55:19,866 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-03-05 19:55:19,866 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-05 19:55:19,866 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-05 19:55:19,866 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-03-05 19:55:19,866 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-05 19:55:19,868 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-05 19:55:19,868 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-05 19:55:19,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-05 19:55:19,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:55:19,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:55:19,868 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-05 19:55:19,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-05 19:55:19,868 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:55:19,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:55:19,869 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:55:19,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:55:19,869 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:19,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-05 19:55:19,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-05 19:55:19,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:55:19,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-05 19:55:19,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:19,870 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:19,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:55:19,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-05 19:55:19,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:19,872 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:55:19,872 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:19,872 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:55:19,872 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:55:19,872 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:55:19,872 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-03-05 19:55:19,872 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-05 19:55:19,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:55:19,872 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-05 19:55:19,872 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-05 19:55:19,872 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:19,872 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-05 19:55:19,872 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-05 19:55:19,872 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-05 19:55:19,872 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-03-05 19:55:19,872 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-05 19:55:19,872 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:55:19,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:55:19,972 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-05 19:55:19,973 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-05 19:55:19,974 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,,1678046116082.daa57ebba7c140c319a2f6facc725e4a. with WAL disabled. Data may be lost in the event of a crash. 2023-03-05 19:55:19,974 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,1,1678046116082.942afd2d0a4ceef025fd5a607956d53d. with WAL disabled. Data may be lost in the event of a crash. 2023-03-05 19:55:19,975 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,2,1678046116082.25af79a83c9b248c956b57fc8507ead6. with WAL disabled. Data may be lost in the event of a crash. 2023-03-05 19:55:19,977 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-05 19:55:19,979 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(143): Procedure testCompactionWithHFileLink was in running list but was completed. Accepting new attempt. 2023-03-05 19:55:19,979 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-03-05 19:55:19,979 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-03-05 19:55:19,980 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:55:19,980 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-03-05 19:55:19,980 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-03-05 19:55:19,980 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-05 19:55:19,980 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-05 19:55:19,982 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:55:19,982 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:19,982 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:55:19,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:55:19,982 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:19,982 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-05 19:55:19,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-05 19:55:19,982 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-05 19:55:19,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-05 19:55:19,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-05 19:55:19,983 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-03-05 19:55:19,983 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testCompactionWithHFileLink is still present, removing 2023-03-05 19:55:19,983 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-03-05 19:55:19,983 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-03-05 19:55:19,983 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:55:19,983 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-03-05 19:55:19,984 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1678046116082.942afd2d0a4ceef025fd5a607956d53d. 2023-03-05 19:55:19,984 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1678046116082.942afd2d0a4ceef025fd5a607956d53d. started... 2023-03-05 19:55:19,984 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1678046116082.25af79a83c9b248c956b57fc8507ead6. 2023-03-05 19:55:19,984 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1678046116082.25af79a83c9b248c956b57fc8507ead6. started... 2023-03-05 19:55:19,984 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 942afd2d0a4ceef025fd5a607956d53d 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-03-05 19:55:19,984 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 25af79a83c9b248c956b57fc8507ead6 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-03-05 19:55:19,989 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-05 19:55:19,990 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-05 19:55:19,990 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1678046116082.daa57ebba7c140c319a2f6facc725e4a. 2023-03-05 19:55:19,990 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1678046116082.daa57ebba7c140c319a2f6facc725e4a. started... 2023-03-05 19:55:19,990 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing daa57ebba7c140c319a2f6facc725e4a 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-03-05 19:55:20,414 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303058825f16fb4854d6eb896c400e4d85db8 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202303058825f16fb4854d6eb896c400e4d85db8 2023-03-05 19:55:20,414 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230305fae8652a174c472c8be7df36b3f66be3 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230305fae8652a174c472c8be7df36b3f66be3 2023-03-05 19:55:20,418 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230305f43d564866564d9e939ce4a0e9ba6b2f to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230305f43d564866564d9e939ce4a0e9ba6b2f 2023-03-05 19:55:20,420 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/.tmp/family1/dc8108fdcc6a4a0a9720b3e2336cb620 2023-03-05 19:55:20,820 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/.tmp/family1/24f57a619907417b8e8a0222c800bc07 2023-03-05 19:55:20,823 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/.tmp/family1/f6b69932b5da4e0d9155ef905eacf40c 2023-03-05 19:55:20,833 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303050699e767fa524db5b06d769d9b9ebc57 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202303050699e767fa524db5b06d769d9b9ebc57 2023-03-05 19:55:21,235 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303058a1c1fcb520745f8bb661ce5d25985d3 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202303058a1c1fcb520745f8bb661ce5d25985d3 2023-03-05 19:55:21,235 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230305b278cd9c1bc2498a99ea539cbb9fc6a0 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230305b278cd9c1bc2498a99ea539cbb9fc6a0 2023-03-05 19:55:21,238 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/.tmp/family2/af93404d81154179a4c27fce0e335552 2023-03-05 19:55:21,242 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/.tmp/family1/dc8108fdcc6a4a0a9720b3e2336cb620 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family1/dc8108fdcc6a4a0a9720b3e2336cb620 2023-03-05 19:55:21,246 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family1/dc8108fdcc6a4a0a9720b3e2336cb620, entries=4, sequenceid=9, filesize=5.4 K 2023-03-05 19:55:21,246 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/.tmp/family2/af93404d81154179a4c27fce0e335552 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family2/af93404d81154179a4c27fce0e335552 2023-03-05 19:55:21,249 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family2/af93404d81154179a4c27fce0e335552, entries=2, sequenceid=9, filesize=5.1 K 2023-03-05 19:55:21,250 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 942afd2d0a4ceef025fd5a607956d53d in 1266ms, sequenceid=9, compaction requested=false 2023-03-05 19:55:21,250 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 942afd2d0a4ceef025fd5a607956d53d: 2023-03-05 19:55:21,250 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1678046116082.942afd2d0a4ceef025fd5a607956d53d. 2023-03-05 19:55:21,250 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-05 19:55:21,302 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-05 19:55:21,642 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/.tmp/family2/e2b047fb6d65462d939ae96ac7375619 2023-03-05 19:55:21,642 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/.tmp/family2/c2d4bdd0af534bf8b270644df2f98eaf 2023-03-05 19:55:21,646 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/.tmp/family1/f6b69932b5da4e0d9155ef905eacf40c as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family1/f6b69932b5da4e0d9155ef905eacf40c 2023-03-05 19:55:21,646 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/.tmp/family1/24f57a619907417b8e8a0222c800bc07 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family1/24f57a619907417b8e8a0222c800bc07 2023-03-05 19:55:21,650 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family1/f6b69932b5da4e0d9155ef905eacf40c, entries=4, sequenceid=9, filesize=5.4 K 2023-03-05 19:55:21,650 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family1/24f57a619907417b8e8a0222c800bc07, entries=4, sequenceid=9, filesize=5.4 K 2023-03-05 19:55:21,650 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/.tmp/family2/c2d4bdd0af534bf8b270644df2f98eaf as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family2/c2d4bdd0af534bf8b270644df2f98eaf 2023-03-05 19:55:21,651 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/.tmp/family2/e2b047fb6d65462d939ae96ac7375619 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family2/e2b047fb6d65462d939ae96ac7375619 2023-03-05 19:55:21,654 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family2/c2d4bdd0af534bf8b270644df2f98eaf, entries=2, sequenceid=9, filesize=5.1 K 2023-03-05 19:55:21,654 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family2/e2b047fb6d65462d939ae96ac7375619, entries=2, sequenceid=9, filesize=5.1 K 2023-03-05 19:55:21,655 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for daa57ebba7c140c319a2f6facc725e4a in 1665ms, sequenceid=9, compaction requested=false 2023-03-05 19:55:21,655 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 25af79a83c9b248c956b57fc8507ead6 in 1671ms, sequenceid=9, compaction requested=false 2023-03-05 19:55:21,655 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for daa57ebba7c140c319a2f6facc725e4a: 2023-03-05 19:55:21,655 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 25af79a83c9b248c956b57fc8507ead6: 2023-03-05 19:55:21,655 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1678046116082.daa57ebba7c140c319a2f6facc725e4a. 2023-03-05 19:55:21,655 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1678046116082.25af79a83c9b248c956b57fc8507ead6. 2023-03-05 19:55:21,655 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-05 19:55:21,655 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-05 19:55:21,655 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-05 19:55:21,655 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:21,655 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-03-05 19:55:21,655 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-03-05 19:55:21,657 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:21,657 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-05 19:55:21,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:21,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:55:21,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:55:21,657 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-05 19:55:21,657 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-05 19:55:21,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:55:21,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:55:21,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-05 19:55:21,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:21,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:55:21,658 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-03-05 19:55:21,658 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5398def8[Count = 0] remaining members to acquire global barrier 2023-03-05 19:55:21,658 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-03-05 19:55:21,658 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-05 19:55:21,659 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-05 19:55:21,659 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-05 19:55:21,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-05 19:55:21,660 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-03-05 19:55:21,660 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-03-05 19:55:21,660 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,37037,1678046018983' in zk 2023-03-05 19:55:21,660 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:21,660 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-05 19:55:21,661 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:21,661 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-03-05 19:55:21,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:21,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:55:21,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:55:21,661 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:55:21,661 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-03-05 19:55:21,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:55:21,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:55:21,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-05 19:55:21,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:21,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:55:21,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-05 19:55:21,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:21,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,37037,1678046018983': 2023-03-05 19:55:21,663 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-03-05 19:55:21,663 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-03-05 19:55:21,663 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-05 19:55:21,663 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-05 19:55:21,663 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-03-05 19:55:21,663 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-05 19:55:21,666 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-05 19:55:21,666 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-05 19:55:21,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-05 19:55:21,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:55:21,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:55:21,666 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-05 19:55:21,666 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:55:21,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-05 19:55:21,667 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:55:21,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:55:21,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:55:21,667 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:21,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-05 19:55:21,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-05 19:55:21,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:55:21,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-05 19:55:21,668 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:21,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:21,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:55:21,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-05 19:55:21,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:21,670 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:21,670 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:55:21,670 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-05 19:55:21,670 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:55:21,670 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-05 19:55:21,670 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-03-05 19:55:21,670 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-05 19:55:21,670 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:55:21,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:55:21,670 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:55:21,670 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink because node does not exist (not an error) 2023-03-05 19:55:21,670 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:21,671 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-05 19:55:21,671 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-05 19:55:21,671 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-03-05 19:55:21,671 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:55:21,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:55:21,671 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-05 19:55:21,671 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-05 19:55:21,771 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-05 19:55:21,772 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-05 19:55:21,773 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,,1678046116082.daa57ebba7c140c319a2f6facc725e4a. with WAL disabled. Data may be lost in the event of a crash. 2023-03-05 19:55:21,773 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,1,1678046116082.942afd2d0a4ceef025fd5a607956d53d. with WAL disabled. Data may be lost in the event of a crash. 2023-03-05 19:55:21,774 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,2,1678046116082.25af79a83c9b248c956b57fc8507ead6. with WAL disabled. Data may be lost in the event of a crash. 2023-03-05 19:55:21,776 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-05 19:55:21,778 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(143): Procedure testCompactionWithHFileLink was in running list but was completed. Accepting new attempt. 2023-03-05 19:55:21,778 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-03-05 19:55:21,778 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-03-05 19:55:21,778 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:55:21,778 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-03-05 19:55:21,778 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-03-05 19:55:21,778 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-05 19:55:21,778 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-05 19:55:21,780 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:55:21,780 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:21,780 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:55:21,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:55:21,780 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:21,780 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-05 19:55:21,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-05 19:55:21,781 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-05 19:55:21,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-05 19:55:21,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-05 19:55:21,781 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-03-05 19:55:21,781 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testCompactionWithHFileLink is still present, removing 2023-03-05 19:55:21,781 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-03-05 19:55:21,781 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-03-05 19:55:21,781 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:55:21,781 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-03-05 19:55:21,792 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1678046116082.942afd2d0a4ceef025fd5a607956d53d. 2023-03-05 19:55:21,792 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1678046116082.942afd2d0a4ceef025fd5a607956d53d. started... 2023-03-05 19:55:21,792 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-05 19:55:21,792 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-05 19:55:21,792 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1678046116082.25af79a83c9b248c956b57fc8507ead6. 2023-03-05 19:55:21,792 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1678046116082.25af79a83c9b248c956b57fc8507ead6. started... 2023-03-05 19:55:21,792 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 942afd2d0a4ceef025fd5a607956d53d 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-03-05 19:55:21,792 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1678046116082.daa57ebba7c140c319a2f6facc725e4a. 2023-03-05 19:55:21,792 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 25af79a83c9b248c956b57fc8507ead6 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-03-05 19:55:21,792 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1678046116082.daa57ebba7c140c319a2f6facc725e4a. started... 2023-03-05 19:55:21,792 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing daa57ebba7c140c319a2f6facc725e4a 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-03-05 19:55:22,222 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230305da73569395114dad837fdbde0dcbba50 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230305da73569395114dad837fdbde0dcbba50 2023-03-05 19:55:22,224 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303056d7220390ee448d6a463d2fc4c42f4da to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202303056d7220390ee448d6a463d2fc4c42f4da 2023-03-05 19:55:22,227 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303058002b27e00b04b36af1ade5009390107 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202303058002b27e00b04b36af1ade5009390107 2023-03-05 19:55:22,630 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/.tmp/family1/43db0ea3bd404e2fa10949ad9df289b4 2023-03-05 19:55:22,631 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/.tmp/family1/3004199e82364ee9abaf18f0f843fc76 2023-03-05 19:55:22,633 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/.tmp/family1/c88f1bd396f4409ca38794d504f5c156 2023-03-05 19:55:23,047 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230305e700388d6bf8428c86eb8d457964da6e to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230305e700388d6bf8428c86eb8d457964da6e 2023-03-05 19:55:23,048 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230305dba00f651aab47a28da5218ebaba00df to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230305dba00f651aab47a28da5218ebaba00df 2023-03-05 19:55:23,048 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023030550ddbcf7ec204ae2a27dd1556aa58a72 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023030550ddbcf7ec204ae2a27dd1556aa58a72 2023-03-05 19:55:23,058 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/.tmp/family2/2c3de0e4dce5444ca51da5f66eaf06a1 2023-03-05 19:55:23,061 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/.tmp/family1/c88f1bd396f4409ca38794d504f5c156 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family1/c88f1bd396f4409ca38794d504f5c156 2023-03-05 19:55:23,064 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family1/c88f1bd396f4409ca38794d504f5c156, entries=4, sequenceid=13, filesize=5.4 K 2023-03-05 19:55:23,064 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/.tmp/family2/2c3de0e4dce5444ca51da5f66eaf06a1 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family2/2c3de0e4dce5444ca51da5f66eaf06a1 2023-03-05 19:55:23,067 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family2/2c3de0e4dce5444ca51da5f66eaf06a1, entries=2, sequenceid=13, filesize=5.1 K 2023-03-05 19:55:23,068 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 942afd2d0a4ceef025fd5a607956d53d in 1275ms, sequenceid=13, compaction requested=true 2023-03-05 19:55:23,068 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 942afd2d0a4ceef025fd5a607956d53d: 2023-03-05 19:55:23,068 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1678046116082.942afd2d0a4ceef025fd5a607956d53d. 2023-03-05 19:55:23,068 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-05 19:55:23,457 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/.tmp/family2/1dfce094a6164169b84ae1182f0260e1 2023-03-05 19:55:23,458 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/.tmp/family2/13ca0708b05643899307a03f0fcc261a 2023-03-05 19:55:23,461 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/.tmp/family1/3004199e82364ee9abaf18f0f843fc76 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family1/3004199e82364ee9abaf18f0f843fc76 2023-03-05 19:55:23,462 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/.tmp/family1/43db0ea3bd404e2fa10949ad9df289b4 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family1/43db0ea3bd404e2fa10949ad9df289b4 2023-03-05 19:55:23,465 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family1/3004199e82364ee9abaf18f0f843fc76, entries=4, sequenceid=13, filesize=5.4 K 2023-03-05 19:55:23,465 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family1/43db0ea3bd404e2fa10949ad9df289b4, entries=4, sequenceid=13, filesize=5.4 K 2023-03-05 19:55:23,466 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/.tmp/family2/1dfce094a6164169b84ae1182f0260e1 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family2/1dfce094a6164169b84ae1182f0260e1 2023-03-05 19:55:23,466 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/.tmp/family2/13ca0708b05643899307a03f0fcc261a as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family2/13ca0708b05643899307a03f0fcc261a 2023-03-05 19:55:23,469 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family2/13ca0708b05643899307a03f0fcc261a, entries=2, sequenceid=13, filesize=5.1 K 2023-03-05 19:55:23,469 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family2/1dfce094a6164169b84ae1182f0260e1, entries=2, sequenceid=13, filesize=5.1 K 2023-03-05 19:55:23,470 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for daa57ebba7c140c319a2f6facc725e4a in 1678ms, sequenceid=13, compaction requested=true 2023-03-05 19:55:23,470 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 25af79a83c9b248c956b57fc8507ead6 in 1678ms, sequenceid=13, compaction requested=true 2023-03-05 19:55:23,470 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for daa57ebba7c140c319a2f6facc725e4a: 2023-03-05 19:55:23,470 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 25af79a83c9b248c956b57fc8507ead6: 2023-03-05 19:55:23,470 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1678046116082.25af79a83c9b248c956b57fc8507ead6. 2023-03-05 19:55:23,470 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1678046116082.daa57ebba7c140c319a2f6facc725e4a. 2023-03-05 19:55:23,470 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-05 19:55:23,470 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-05 19:55:23,470 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-05 19:55:23,470 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:23,470 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-03-05 19:55:23,470 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-03-05 19:55:23,472 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:23,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:23,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:55:23,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:55:23,472 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-05 19:55:23,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:55:23,473 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-05 19:55:23,473 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-05 19:55:23,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:55:23,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-05 19:55:23,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:23,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:55:23,474 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-03-05 19:55:23,474 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6909ff7e[Count = 0] remaining members to acquire global barrier 2023-03-05 19:55:23,474 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-03-05 19:55:23,474 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-05 19:55:23,479 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-05 19:55:23,479 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-05 19:55:23,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-05 19:55:23,479 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:23,479 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-05 19:55:23,479 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-03-05 19:55:23,479 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-03-05 19:55:23,479 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,37037,1678046018983' in zk 2023-03-05 19:55:23,481 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-03-05 19:55:23,481 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:23,481 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:55:23,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:23,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:55:23,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:55:23,481 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-03-05 19:55:23,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:55:23,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:55:23,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-05 19:55:23,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:23,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:55:23,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-05 19:55:23,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:23,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,37037,1678046018983': 2023-03-05 19:55:23,483 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-03-05 19:55:23,483 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-03-05 19:55:23,483 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-05 19:55:23,483 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-05 19:55:23,483 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-03-05 19:55:23,483 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-05 19:55:23,487 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-05 19:55:23,487 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-05 19:55:23,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-05 19:55:23,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:55:23,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:55:23,487 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-05 19:55:23,487 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:55:23,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-05 19:55:23,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:55:23,488 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:23,488 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:55:23,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:55:23,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-05 19:55:23,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-05 19:55:23,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:55:23,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-05 19:55:23,489 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:23,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:23,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:55:23,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-05 19:55:23,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:23,492 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:23,492 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:55:23,492 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:55:23,492 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink because node does not exist (not an error) 2023-03-05 19:55:23,492 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-05 19:55:23,492 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:55:23,492 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-03-05 19:55:23,492 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:55:23,492 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-05 19:55:23,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:55:23,492 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-05 19:55:23,492 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:23,492 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-05 19:55:23,492 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-05 19:55:23,492 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-05 19:55:23,492 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-03-05 19:55:23,493 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-05 19:55:23,493 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:55:23,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:55:23,593 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-05 19:55:23,593 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-05 19:55:23,595 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,,1678046116082.daa57ebba7c140c319a2f6facc725e4a. with WAL disabled. Data may be lost in the event of a crash. 2023-03-05 19:55:23,595 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,1,1678046116082.942afd2d0a4ceef025fd5a607956d53d. with WAL disabled. Data may be lost in the event of a crash. 2023-03-05 19:55:23,595 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,2,1678046116082.25af79a83c9b248c956b57fc8507ead6. with WAL disabled. Data may be lost in the event of a crash. 2023-03-05 19:55:23,597 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-05 19:55:23,599 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(143): Procedure testCompactionWithHFileLink was in running list but was completed. Accepting new attempt. 2023-03-05 19:55:23,599 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-03-05 19:55:23,599 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-03-05 19:55:23,599 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:55:23,599 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-03-05 19:55:23,599 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-03-05 19:55:23,600 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-05 19:55:23,600 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-05 19:55:23,601 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:23,601 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:55:23,601 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:55:23,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:55:23,602 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:23,602 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-05 19:55:23,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-05 19:55:23,602 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-05 19:55:23,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-05 19:55:23,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-05 19:55:23,602 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-03-05 19:55:23,602 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testCompactionWithHFileLink is still present, removing 2023-03-05 19:55:23,602 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-03-05 19:55:23,602 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-03-05 19:55:23,603 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:55:23,603 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-03-05 19:55:23,603 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1678046116082.942afd2d0a4ceef025fd5a607956d53d. 2023-03-05 19:55:23,603 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1678046116082.942afd2d0a4ceef025fd5a607956d53d. started... 2023-03-05 19:55:23,603 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1678046116082.25af79a83c9b248c956b57fc8507ead6. 2023-03-05 19:55:23,603 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-05 19:55:23,603 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1678046116082.daa57ebba7c140c319a2f6facc725e4a. 2023-03-05 19:55:23,603 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1678046116082.25af79a83c9b248c956b57fc8507ead6. started... 2023-03-05 19:55:23,603 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1678046116082.daa57ebba7c140c319a2f6facc725e4a. started... 2023-03-05 19:55:23,603 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-05 19:55:23,603 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 942afd2d0a4ceef025fd5a607956d53d 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-03-05 19:55:23,603 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing daa57ebba7c140c319a2f6facc725e4a 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-03-05 19:55:23,603 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 25af79a83c9b248c956b57fc8507ead6 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-03-05 19:55:23,617 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303057212e8b9c498437fb3c20726fe55a714 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202303057212e8b9c498437fb3c20726fe55a714 2023-03-05 19:55:23,622 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/.tmp/family1/6b8608e70a864d6e8a4e55c85261adfe 2023-03-05 19:55:23,622 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303056a09d4fe8b054b179c0313b89fbc21ce to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202303056a09d4fe8b054b179c0313b89fbc21ce 2023-03-05 19:55:24,017 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023030501cbec120485489fa59d0ed556ae93f6 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023030501cbec120485489fa59d0ed556ae93f6 2023-03-05 19:55:24,027 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/.tmp/family1/e02bc7cebc7e44af831019cee45cd29b 2023-03-05 19:55:24,034 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230305a54669736605415e9bffb71bcfa30d8e to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230305a54669736605415e9bffb71bcfa30d8e 2023-03-05 19:55:24,422 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/.tmp/family1/8139f54670e14aa3a572f9bdb2477108 2023-03-05 19:55:24,439 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/.tmp/family2/11fd411913d44c5187ed3f273ed56bdd 2023-03-05 19:55:24,440 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230305b7c7c8f3cd6343f9870b0f76ea40a2c6 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230305b7c7c8f3cd6343f9870b0f76ea40a2c6 2023-03-05 19:55:24,443 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/.tmp/family1/6b8608e70a864d6e8a4e55c85261adfe as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family1/6b8608e70a864d6e8a4e55c85261adfe 2023-03-05 19:55:24,447 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family1/6b8608e70a864d6e8a4e55c85261adfe, entries=4, sequenceid=17, filesize=5.4 K 2023-03-05 19:55:24,447 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/.tmp/family2/11fd411913d44c5187ed3f273ed56bdd as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family2/11fd411913d44c5187ed3f273ed56bdd 2023-03-05 19:55:24,451 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family2/11fd411913d44c5187ed3f273ed56bdd, entries=2, sequenceid=17, filesize=5.1 K 2023-03-05 19:55:24,451 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 25af79a83c9b248c956b57fc8507ead6 in 848ms, sequenceid=17, compaction requested=true 2023-03-05 19:55:24,451 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 25af79a83c9b248c956b57fc8507ead6: 2023-03-05 19:55:24,451 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1678046116082.25af79a83c9b248c956b57fc8507ead6. 2023-03-05 19:55:24,452 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-05 19:55:24,835 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303056312607d0a274d938258ee9920246856 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202303056312607d0a274d938258ee9920246856 2023-03-05 19:55:24,845 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/.tmp/family2/9173733bfa77411fa7038a412b2f97e1 2023-03-05 19:55:24,849 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/.tmp/family1/e02bc7cebc7e44af831019cee45cd29b as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family1/e02bc7cebc7e44af831019cee45cd29b 2023-03-05 19:55:24,852 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family1/e02bc7cebc7e44af831019cee45cd29b, entries=4, sequenceid=17, filesize=5.4 K 2023-03-05 19:55:24,852 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/.tmp/family2/9173733bfa77411fa7038a412b2f97e1 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family2/9173733bfa77411fa7038a412b2f97e1 2023-03-05 19:55:24,855 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family2/9173733bfa77411fa7038a412b2f97e1, entries=2, sequenceid=17, filesize=5.1 K 2023-03-05 19:55:24,856 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for daa57ebba7c140c319a2f6facc725e4a in 1253ms, sequenceid=17, compaction requested=true 2023-03-05 19:55:24,856 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for daa57ebba7c140c319a2f6facc725e4a: 2023-03-05 19:55:24,856 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1678046116082.daa57ebba7c140c319a2f6facc725e4a. 2023-03-05 19:55:24,856 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-05 19:55:25,240 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/.tmp/family2/55d239cb7531440088e28c0c9e6143d3 2023-03-05 19:55:25,244 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/.tmp/family1/8139f54670e14aa3a572f9bdb2477108 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family1/8139f54670e14aa3a572f9bdb2477108 2023-03-05 19:55:25,248 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family1/8139f54670e14aa3a572f9bdb2477108, entries=4, sequenceid=17, filesize=5.4 K 2023-03-05 19:55:25,248 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/.tmp/family2/55d239cb7531440088e28c0c9e6143d3 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family2/55d239cb7531440088e28c0c9e6143d3 2023-03-05 19:55:25,252 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family2/55d239cb7531440088e28c0c9e6143d3, entries=2, sequenceid=17, filesize=5.1 K 2023-03-05 19:55:25,252 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 942afd2d0a4ceef025fd5a607956d53d in 1649ms, sequenceid=17, compaction requested=true 2023-03-05 19:55:25,252 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 942afd2d0a4ceef025fd5a607956d53d: 2023-03-05 19:55:25,252 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1678046116082.942afd2d0a4ceef025fd5a607956d53d. 2023-03-05 19:55:25,252 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-05 19:55:25,253 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-05 19:55:25,253 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:25,253 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-03-05 19:55:25,253 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-03-05 19:55:25,254 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-05 19:55:25,254 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:25,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:25,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:55:25,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:55:25,255 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-05 19:55:25,255 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-05 19:55:25,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:55:25,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:55:25,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-05 19:55:25,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:25,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:55:25,256 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-03-05 19:55:25,256 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@77c5f399[Count = 0] remaining members to acquire global barrier 2023-03-05 19:55:25,256 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-03-05 19:55:25,256 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-05 19:55:25,257 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-05 19:55:25,257 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-05 19:55:25,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-05 19:55:25,257 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-03-05 19:55:25,257 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-03-05 19:55:25,257 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,37037,1678046018983' in zk 2023-03-05 19:55:25,257 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:25,257 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-05 19:55:25,260 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-03-05 19:55:25,260 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:55:25,260 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:25,260 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-03-05 19:55:25,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:25,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:55:25,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:55:25,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:55:25,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:55:25,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-05 19:55:25,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:25,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:55:25,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-05 19:55:25,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:25,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,37037,1678046018983': 2023-03-05 19:55:25,262 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-03-05 19:55:25,262 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-05 19:55:25,262 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-05 19:55:25,262 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-03-05 19:55:25,262 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-05 19:55:25,262 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-03-05 19:55:25,264 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-05 19:55:25,264 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-05 19:55:25,264 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-05 19:55:25,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-05 19:55:25,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:55:25,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:55:25,264 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:55:25,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-05 19:55:25,264 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:25,264 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:55:25,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:55:25,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:55:25,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-05 19:55:25,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-05 19:55:25,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:55:25,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-05 19:55:25,265 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:25,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:25,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:55:25,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-05 19:55:25,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:25,267 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:25,267 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:55:25,267 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-05 19:55:25,267 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-03-05 19:55:25,267 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-05 19:55:25,267 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-05 19:55:25,267 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:25,267 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-05 19:55:25,267 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-05 19:55:25,267 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-05 19:55:25,267 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink because node does not exist (not an error) 2023-03-05 19:55:25,267 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:55:25,267 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:55:25,267 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:55:25,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:55:25,267 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-03-05 19:55:25,268 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-05 19:55:25,268 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:55:25,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:55:25,368 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-05 19:55:25,368 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-05 19:55:25,378 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1580): Client=jenkins//172.31.14.131 snapshot request for:{ ss=snaptb-1678046125369 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-03-05 19:55:25,379 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] snapshot.SnapshotDescriptionUtils(316): Creation time not specified, setting to:1678046125378 (current time:1678046125379). 2023-03-05 19:55:25,379 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] snapshot.SnapshotDescriptionUtils(332): Snapshot current TTL value: 0 resetting it to default value: 0 2023-03-05 19:55:25,379 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] zookeeper.ReadOnlyZKClient(139): Connect 0x151606c0 to 127.0.0.1:62957 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-05 19:55:25,385 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4485aef6, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-05 19:55:25,389 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-05 19:55:25,392 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:57872, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-05 19:55:25,392 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x151606c0 to 127.0.0.1:62957 2023-03-05 19:55:25,392 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-05 19:55:25,393 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] snapshot.SnapshotManager(601): No existing snapshot, attempting snapshot... 2023-03-05 19:55:25,393 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] snapshot.SnapshotManager(648): Table enabled, starting distributed snapshots for { ss=snaptb-1678046125369 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-03-05 19:55:25,401 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure2.ProcedureExecutor(1029): Stored pid=86, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-03-05 19:55:25,402 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=86, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-03-05 19:55:25,402 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=86, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE; timeout=600000, timestamp=1678046725402 2023-03-05 19:55:25,403 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] snapshot.SnapshotManager(653): Started snapshot: { ss=snaptb-1678046125369 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-03-05 19:55:25,403 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(174): Running FLUSH table snapshot snaptb-1678046125369 C_M_SNAPSHOT_TABLE on table testCompactionWithHFileLink 2023-03-05 19:55:25,404 DEBUG [PEWorker-3] locking.LockProcedure(242): UNLOCKED pid=86, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-03-05 19:55:25,404 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] procedure2.ProcedureExecutor(1029): Stored pid=87, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED 2023-03-05 19:55:25,405 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=86, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE in 3 msec 2023-03-05 19:55:25,405 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=87, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED 2023-03-05 19:55:25,405 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=87, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED; timeout=600000, timestamp=1678046725405 2023-03-05 19:55:25,408 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2418): Waiting a max of 300000 ms for snapshot '{ ss=snaptb-1678046125369 table=testCompactionWithHFileLink type=FLUSH ttl=0 }'' to complete. (max 300000 ms per retry) 2023-03-05 19:55:25,409 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2428): (#1) Sleeping: 100ms while waiting for snapshot completion. 2023-03-05 19:55:25,509 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-05 19:55:25,511 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1678046125369 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-05 19:55:25,513 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] snapshot.SnapshotManager(404): Snapshoting '{ ss=snaptb-1678046125369 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-03-05 19:55:25,513 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2428): (#2) Sleeping: 200ms while waiting for snapshot completion. 2023-03-05 19:55:25,713 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-05 19:55:25,714 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1678046125369 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-05 19:55:25,714 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] snapshot.SnapshotManager(404): Snapshoting '{ ss=snaptb-1678046125369 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-03-05 19:55:25,714 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2428): (#3) Sleeping: 300ms while waiting for snapshot completion. 2023-03-05 19:55:25,817 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] procedure.ProcedureCoordinator(165): Submitting procedure snaptb-1678046125369 2023-03-05 19:55:25,817 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'snaptb-1678046125369' 2023-03-05 19:55:25,817 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 300000 ms 2023-03-05 19:55:25,817 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'snaptb-1678046125369' starting 'acquire' 2023-03-05 19:55:25,817 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'snaptb-1678046125369', kicking off acquire phase on members. 2023-03-05 19:55:25,817 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/abort/snaptb-1678046125369 2023-03-05 19:55:25,818 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/online-snapshot/acquired/snaptb-1678046125369 2023-03-05 19:55:25,819 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/online-snapshot/acquired/snaptb-1678046125369/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:25,819 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/acquired 2023-03-05 19:55:25,819 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/online-snapshot/acquired 2023-03-05 19:55:25,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-03-05 19:55:25,819 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/acquired/snaptb-1678046125369/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:25,819 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-05 19:55:25,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/online-snapshot/acquired/snaptb-1678046125369 2023-03-05 19:55:25,820 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/abort/snaptb-1678046125369 2023-03-05 19:55:25,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 68 2023-03-05 19:55:25,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/online-snapshot/acquired/snaptb-1678046125369 2023-03-05 19:55:25,820 DEBUG [zk-event-processor-pool-0] snapshot.RegionServerSnapshotManager(175): Launching subprocedure for snapshot snaptb-1678046125369 from table testCompactionWithHFileLink type FLUSH 2023-03-05 19:55:25,822 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:snaptb-1678046125369 2023-03-05 19:55:25,822 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'snaptb-1678046125369' with timeout 300000ms 2023-03-05 19:55:25,822 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 300000 ms 2023-03-05 19:55:25,823 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'snaptb-1678046125369' starting 'acquire' stage 2023-03-05 19:55:25,823 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'snaptb-1678046125369' locally acquired 2023-03-05 19:55:25,823 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure (snaptb-1678046125369) in zk 2023-03-05 19:55:25,824 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/acquired/snaptb-1678046125369/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:25,824 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/online-snapshot/reached/snaptb-1678046125369 2023-03-05 19:55:25,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/online-snapshot/acquired/snaptb-1678046125369/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:25,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:55:25,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/online-snapshot 2023-03-05 19:55:25,825 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/reached/snaptb-1678046125369 2023-03-05 19:55:25,825 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'snaptb-1678046125369' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-05 19:55:25,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:55:25,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:55:25,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1678046125369 2023-03-05 19:55:25,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:25,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:55:25,826 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure 'snaptb-1678046125369' on coordinator 2023-03-05 19:55:25,826 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@86ddbf2[Count = 0] remaining members to acquire global barrier 2023-03-05 19:55:25,826 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'snaptb-1678046125369' starting 'in-barrier' execution. 2023-03-05 19:55:25,826 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/online-snapshot/reached/snaptb-1678046125369 2023-03-05 19:55:25,827 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1678046125369 2023-03-05 19:55:25,827 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/online-snapshot/reached/snaptb-1678046125369 2023-03-05 19:55:25,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/online-snapshot/reached/snaptb-1678046125369 2023-03-05 19:55:25,827 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'snaptb-1678046125369' received 'reached' from coordinator. 2023-03-05 19:55:25,827 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/reached/snaptb-1678046125369/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:25,827 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-05 19:55:25,828 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(97): Starting snapshot operation on testCompactionWithHFileLink,1,1678046116082.942afd2d0a4ceef025fd5a607956d53d. 2023-03-05 19:55:25,828 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(97): Starting snapshot operation on testCompactionWithHFileLink,2,1678046116082.25af79a83c9b248c956b57fc8507ead6. 2023-03-05 19:55:25,828 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(97): Starting snapshot operation on testCompactionWithHFileLink,,1678046116082.daa57ebba7c140c319a2f6facc725e4a. 2023-03-05 19:55:25,828 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] snapshot.FlushSnapshotSubprocedure(170): Flush Snapshot Tasks submitted for 3 regions 2023-03-05 19:55:25,828 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(301): Waiting for local region snapshots to finish. 2023-03-05 19:55:25,829 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(110): Flush Snapshotting region testCompactionWithHFileLink,2,1678046116082.25af79a83c9b248c956b57fc8507ead6. started... 2023-03-05 19:55:25,829 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(110): Flush Snapshotting region testCompactionWithHFileLink,1,1678046116082.942afd2d0a4ceef025fd5a607956d53d. started... 2023-03-05 19:55:25,829 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(110): Flush Snapshotting region testCompactionWithHFileLink,,1678046116082.daa57ebba7c140c319a2f6facc725e4a. started... 2023-03-05 19:55:25,829 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-snapshot-pool-1] regionserver.HRegion(2446): Flush status journal for 25af79a83c9b248c956b57fc8507ead6: 2023-03-05 19:55:25,829 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-snapshot-pool-0] regionserver.HRegion(2446): Flush status journal for 942afd2d0a4ceef025fd5a607956d53d: 2023-03-05 19:55:25,829 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-snapshot-pool-2] regionserver.HRegion(2446): Flush status journal for daa57ebba7c140c319a2f6facc725e4a: 2023-03-05 19:55:25,830 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-snapshot-pool-1] snapshot.SnapshotManifest(238): Storing 'testCompactionWithHFileLink,2,1678046116082.25af79a83c9b248c956b57fc8507ead6.' region-info for snapshot=snaptb-1678046125369 2023-03-05 19:55:25,830 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-snapshot-pool-2] snapshot.SnapshotManifest(238): Storing 'testCompactionWithHFileLink,,1678046116082.daa57ebba7c140c319a2f6facc725e4a.' region-info for snapshot=snaptb-1678046125369 2023-03-05 19:55:25,830 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-snapshot-pool-0] snapshot.SnapshotManifest(238): Storing 'testCompactionWithHFileLink,1,1678046116082.942afd2d0a4ceef025fd5a607956d53d.' region-info for snapshot=snaptb-1678046125369 2023-03-05 19:55:25,979 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-snapshot-pool-0] snapshot.SnapshotManifest(243): Creating references for hfiles 2023-03-05 19:55:25,979 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-snapshot-pool-2] snapshot.SnapshotManifest(243): Creating references for hfiles 2023-03-05 19:55:25,979 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-05 19:55:25,979 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-snapshot-pool-1] snapshot.SnapshotManifest(243): Creating references for hfiles 2023-03-05 19:55:25,979 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(142): Chore: CompactedHFilesCleaner missed its start time 2023-03-05 19:55:25,979 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x6872a4ff to 127.0.0.1:62957 2023-03-05 19:55:25,979 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-05 19:55:25,980 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x46149275 to 127.0.0.1:62957 2023-03-05 19:55:25,980 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-05 19:55:25,980 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-05 19:55:25,980 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x2951c910 to 127.0.0.1:62957 2023-03-05 19:55:25,980 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-05 19:55:25,980 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-05 19:55:25,980 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x21ece7aa to 127.0.0.1:62957 2023-03-05 19:55:25,980 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-05 19:55:25,984 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-snapshot-pool-0] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family1/2ddb935fe9dc46778a4c450ff10d3c44, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family1/dc8108fdcc6a4a0a9720b3e2336cb620, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family1/c88f1bd396f4409ca38794d504f5c156, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family1/8139f54670e14aa3a572f9bdb2477108] hfiles 2023-03-05 19:55:25,984 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-snapshot-pool-2] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family1/cdcb9dd831794696af2aaf30754840ee, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family1/f6b69932b5da4e0d9155ef905eacf40c, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family1/43db0ea3bd404e2fa10949ad9df289b4, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family1/e02bc7cebc7e44af831019cee45cd29b] hfiles 2023-03-05 19:55:25,984 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family1/2ddb935fe9dc46778a4c450ff10d3c44 for snapshot=snaptb-1678046125369 2023-03-05 19:55:25,984 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family1/cdcb9dd831794696af2aaf30754840ee for snapshot=snaptb-1678046125369 2023-03-05 19:55:25,984 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-snapshot-pool-1] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family1/9a9155a47d074f5f9005ddf6337e173e, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family1/24f57a619907417b8e8a0222c800bc07, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family1/3004199e82364ee9abaf18f0f843fc76, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family1/6b8608e70a864d6e8a4e55c85261adfe] hfiles 2023-03-05 19:55:25,984 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family1/9a9155a47d074f5f9005ddf6337e173e for snapshot=snaptb-1678046125369 2023-03-05 19:55:25,987 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family1/f6b69932b5da4e0d9155ef905eacf40c for snapshot=snaptb-1678046125369 2023-03-05 19:55:25,987 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family1/dc8108fdcc6a4a0a9720b3e2336cb620 for snapshot=snaptb-1678046125369 2023-03-05 19:55:25,987 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family1/24f57a619907417b8e8a0222c800bc07 for snapshot=snaptb-1678046125369 2023-03-05 19:55:25,987 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family1/c88f1bd396f4409ca38794d504f5c156 for snapshot=snaptb-1678046125369 2023-03-05 19:55:25,987 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family1/43db0ea3bd404e2fa10949ad9df289b4 for snapshot=snaptb-1678046125369 2023-03-05 19:55:25,987 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family1/8139f54670e14aa3a572f9bdb2477108 for snapshot=snaptb-1678046125369 2023-03-05 19:55:25,987 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family1/3004199e82364ee9abaf18f0f843fc76 for snapshot=snaptb-1678046125369 2023-03-05 19:55:25,987 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family1/e02bc7cebc7e44af831019cee45cd29b for snapshot=snaptb-1678046125369 2023-03-05 19:55:25,987 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family1/6b8608e70a864d6e8a4e55c85261adfe for snapshot=snaptb-1678046125369 2023-03-05 19:55:25,987 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-snapshot-pool-2] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family2/c42309b45f3243f2a021144945ad30f4, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family2/c2d4bdd0af534bf8b270644df2f98eaf, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family2/13ca0708b05643899307a03f0fcc261a, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family2/9173733bfa77411fa7038a412b2f97e1] hfiles 2023-03-05 19:55:25,987 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-snapshot-pool-1] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family2/22cbfc0fa2af442f976458cd740e398d, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family2/e2b047fb6d65462d939ae96ac7375619, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family2/1dfce094a6164169b84ae1182f0260e1, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family2/11fd411913d44c5187ed3f273ed56bdd] hfiles 2023-03-05 19:55:25,988 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family2/22cbfc0fa2af442f976458cd740e398d for snapshot=snaptb-1678046125369 2023-03-05 19:55:25,987 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-snapshot-pool-0] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family2/9a13755a01ce4e01b7e321e95b43b8cc, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family2/af93404d81154179a4c27fce0e335552, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family2/2c3de0e4dce5444ca51da5f66eaf06a1, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family2/55d239cb7531440088e28c0c9e6143d3] hfiles 2023-03-05 19:55:25,988 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family2/9a13755a01ce4e01b7e321e95b43b8cc for snapshot=snaptb-1678046125369 2023-03-05 19:55:25,988 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family2/e2b047fb6d65462d939ae96ac7375619 for snapshot=snaptb-1678046125369 2023-03-05 19:55:25,987 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family2/c42309b45f3243f2a021144945ad30f4 for snapshot=snaptb-1678046125369 2023-03-05 19:55:25,988 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family2/1dfce094a6164169b84ae1182f0260e1 for snapshot=snaptb-1678046125369 2023-03-05 19:55:25,988 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family2/11fd411913d44c5187ed3f273ed56bdd for snapshot=snaptb-1678046125369 2023-03-05 19:55:25,988 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family2/af93404d81154179a4c27fce0e335552 for snapshot=snaptb-1678046125369 2023-03-05 19:55:25,988 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family2/c2d4bdd0af534bf8b270644df2f98eaf for snapshot=snaptb-1678046125369 2023-03-05 19:55:25,988 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family2/2c3de0e4dce5444ca51da5f66eaf06a1 for snapshot=snaptb-1678046125369 2023-03-05 19:55:25,988 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family2/13ca0708b05643899307a03f0fcc261a for snapshot=snaptb-1678046125369 2023-03-05 19:55:25,988 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family2/55d239cb7531440088e28c0c9e6143d3 for snapshot=snaptb-1678046125369 2023-03-05 19:55:25,988 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family2/9173733bfa77411fa7038a412b2f97e1 for snapshot=snaptb-1678046125369 2023-03-05 19:55:26,014 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-05 19:55:26,015 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1678046125369 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-05 19:55:26,015 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] snapshot.SnapshotManager(404): Snapshoting '{ ss=snaptb-1678046125369 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-03-05 19:55:26,015 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2428): (#4) Sleeping: 500ms while waiting for snapshot completion. 2023-03-05 19:55:26,394 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(137): ... Flush Snapshotting region testCompactionWithHFileLink,1,1678046116082.942afd2d0a4ceef025fd5a607956d53d. completed. 2023-03-05 19:55:26,394 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(137): ... Flush Snapshotting region testCompactionWithHFileLink,2,1678046116082.25af79a83c9b248c956b57fc8507ead6. completed. 2023-03-05 19:55:26,394 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(140): Closing snapshot operation on testCompactionWithHFileLink,2,1678046116082.25af79a83c9b248c956b57fc8507ead6. 2023-03-05 19:55:26,394 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(140): Closing snapshot operation on testCompactionWithHFileLink,1,1678046116082.942afd2d0a4ceef025fd5a607956d53d. 2023-03-05 19:55:26,394 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(137): ... Flush Snapshotting region testCompactionWithHFileLink,,1678046116082.daa57ebba7c140c319a2f6facc725e4a. completed. 2023-03-05 19:55:26,395 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(140): Closing snapshot operation on testCompactionWithHFileLink,,1678046116082.daa57ebba7c140c319a2f6facc725e4a. 2023-03-05 19:55:26,395 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(312): Completed 1/3 local region snapshots. 2023-03-05 19:55:26,395 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(312): Completed 2/3 local region snapshots. 2023-03-05 19:55:26,395 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(312): Completed 3/3 local region snapshots. 2023-03-05 19:55:26,395 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(314): Completed 3 local region snapshots. 2023-03-05 19:55:26,395 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(345): cancelling 0 tasks for snapshot jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:26,395 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'snaptb-1678046125369' locally completed 2023-03-05 19:55:26,395 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'snaptb-1678046125369' completed for member 'jenkins-hbase4.apache.org,37037,1678046018983' in zk 2023-03-05 19:55:26,397 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'snaptb-1678046125369' has notified controller of completion 2023-03-05 19:55:26,397 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1678046125369/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:26,397 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:55:26,397 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'snaptb-1678046125369' completed. 2023-03-05 19:55:26,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/online-snapshot/reached/snaptb-1678046125369/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:26,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:55:26,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/online-snapshot 2023-03-05 19:55:26,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:55:26,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:55:26,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1678046125369 2023-03-05 19:55:26,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:26,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:55:26,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1678046125369 2023-03-05 19:55:26,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:26,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'snaptb-1678046125369' member 'jenkins-hbase4.apache.org,37037,1678046018983': 2023-03-05 19:55:26,399 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' released barrier for procedure'snaptb-1678046125369', counting down latch. Waiting for 0 more 2023-03-05 19:55:26,399 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'snaptb-1678046125369' execution completed 2023-03-05 19:55:26,399 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-05 19:55:26,399 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-05 19:55:26,399 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:snaptb-1678046125369 2023-03-05 19:55:26,399 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure snaptb-1678046125369including nodes /hbase/online-snapshot/acquired /hbase/online-snapshot/reached /hbase/online-snapshot/abort 2023-03-05 19:55:26,401 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/abort/snaptb-1678046125369 2023-03-05 19:55:26,401 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/abort/snaptb-1678046125369 2023-03-05 19:55:26,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/online-snapshot/abort/snaptb-1678046125369 2023-03-05 19:55:26,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:55:26,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/online-snapshot 2023-03-05 19:55:26,401 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/online-snapshot/abort/snaptb-1678046125369 2023-03-05 19:55:26,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/online-snapshot/abort/snaptb-1678046125369 2023-03-05 19:55:26,401 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/abort 2023-03-05 19:55:26,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:55:26,401 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/online-snapshot/abort 2023-03-05 19:55:26,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-03-05 19:55:26,401 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/online-snapshot/acquired/snaptb-1678046125369/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:26,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1678046125369 2023-03-05 19:55:26,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/online-snapshot/abort/snaptb-1678046125369 2023-03-05 19:55:26,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:55:26,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1678046125369 2023-03-05 19:55:26,402 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/online-snapshot/reached/snaptb-1678046125369/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:26,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:26,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:55:26,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1678046125369 2023-03-05 19:55:26,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:26,406 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/acquired/snaptb-1678046125369/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:26,406 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/acquired 2023-03-05 19:55:26,406 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/acquired/snaptb-1678046125369 2023-03-05 19:55:26,406 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:55:26,406 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/acquired/snaptb-1678046125369 2023-03-05 19:55:26,406 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.EnabledTableSnapshotHandler(97): Done waiting - online snapshot for snaptb-1678046125369 2023-03-05 19:55:26,406 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1678046125369/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:26,406 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/abort 2023-03-05 19:55:26,407 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.EnabledTableSnapshotHandler(113): Taking snapshot for mob files in table testCompactionWithHFileLink 2023-03-05 19:55:26,406 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/online-snapshot/acquired 2023-03-05 19:55:26,407 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(191): Storing mob region '{ENCODED => 0e7b6fa9e81bd62588edd7d77d6165e4, NAME => 'testCompactionWithHFileLink,.mob,0.0e7b6fa9e81bd62588edd7d77d6165e4.', STARTKEY => '.mob', ENDKEY => ''}' region-info for snapshot=snaptb-1678046125369 2023-03-05 19:55:26,407 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1678046125369 2023-03-05 19:55:26,407 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(196): Creating references for mob files 2023-03-05 19:55:26,407 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1678046125369 2023-03-05 19:55:26,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-03-05 19:55:26,407 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/abort/snaptb-1678046125369 2023-03-05 19:55:26,407 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/online-snapshot/abort 2023-03-05 19:55:26,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-03-05 19:55:26,408 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023030501cbec120485489fa59d0ed556ae93f6, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230305202e97351c394119851f4075ef789695, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202303058002b27e00b04b36af1ade5009390107, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230305fae8652a174c472c8be7df36b3f66be3, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023030524196f3846324173b9e8c56ba282d604, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202303056d7220390ee448d6a463d2fc4c42f4da, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202303057212e8b9c498437fb3c20726fe55a714, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202303058825f16fb4854d6eb896c400e4d85db8, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202303056a09d4fe8b054b179c0313b89fbc21ce, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230305a33235cee5224842811abef82371aea6, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230305da73569395114dad837fdbde0dcbba50, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230305f43d564866564d9e939ce4a0e9ba6b2f] mob files 2023-03-05 19:55:26,408 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (1/12): hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023030501cbec120485489fa59d0ed556ae93f6 2023-03-05 19:55:26,408 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (2/12): hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230305202e97351c394119851f4075ef789695 2023-03-05 19:55:26,408 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (3/12): hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202303058002b27e00b04b36af1ade5009390107 2023-03-05 19:55:26,408 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (4/12): hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230305fae8652a174c472c8be7df36b3f66be3 2023-03-05 19:55:26,408 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (5/12): hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023030524196f3846324173b9e8c56ba282d604 2023-03-05 19:55:26,408 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (6/12): hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202303056d7220390ee448d6a463d2fc4c42f4da 2023-03-05 19:55:26,408 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (7/12): hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202303057212e8b9c498437fb3c20726fe55a714 2023-03-05 19:55:26,408 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (8/12): hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202303058825f16fb4854d6eb896c400e4d85db8 2023-03-05 19:55:26,408 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (9/12): hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202303056a09d4fe8b054b179c0313b89fbc21ce 2023-03-05 19:55:26,408 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (10/12): hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230305a33235cee5224842811abef82371aea6 2023-03-05 19:55:26,408 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (11/12): hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230305da73569395114dad837fdbde0dcbba50 2023-03-05 19:55:26,408 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (12/12): hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230305f43d564866564d9e939ce4a0e9ba6b2f 2023-03-05 19:55:26,409 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202303050699e767fa524db5b06d769d9b9ebc57, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202303052c52d973fc3d4c1d97d47e99d0c89ee5, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023030550ddbcf7ec204ae2a27dd1556aa58a72, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202303056312607d0a274d938258ee9920246856, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230305a54669736605415e9bffb71bcfa30d8e, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230305b278cd9c1bc2498a99ea539cbb9fc6a0, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230305e700388d6bf8428c86eb8d457964da6e, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230305f7d80ee747a04a6ca514f2b4ec56d1a4, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023030553b3fcde254241d4b99a405451edd672, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202303058a1c1fcb520745f8bb661ce5d25985d3, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230305b7c7c8f3cd6343f9870b0f76ea40a2c6, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230305dba00f651aab47a28da5218ebaba00df] mob files 2023-03-05 19:55:26,409 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (1/12): hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202303050699e767fa524db5b06d769d9b9ebc57 2023-03-05 19:55:26,409 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (2/12): hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202303052c52d973fc3d4c1d97d47e99d0c89ee5 2023-03-05 19:55:26,409 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (3/12): hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023030550ddbcf7ec204ae2a27dd1556aa58a72 2023-03-05 19:55:26,409 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (4/12): hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202303056312607d0a274d938258ee9920246856 2023-03-05 19:55:26,409 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (5/12): hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230305a54669736605415e9bffb71bcfa30d8e 2023-03-05 19:55:26,409 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (6/12): hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230305b278cd9c1bc2498a99ea539cbb9fc6a0 2023-03-05 19:55:26,409 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (7/12): hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230305e700388d6bf8428c86eb8d457964da6e 2023-03-05 19:55:26,409 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (8/12): hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230305f7d80ee747a04a6ca514f2b4ec56d1a4 2023-03-05 19:55:26,409 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (9/12): hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023030553b3fcde254241d4b99a405451edd672 2023-03-05 19:55:26,409 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (10/12): hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202303058a1c1fcb520745f8bb661ce5d25985d3 2023-03-05 19:55:26,409 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (11/12): hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230305b7c7c8f3cd6343f9870b0f76ea40a2c6 2023-03-05 19:55:26,410 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (12/12): hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230305dba00f651aab47a28da5218ebaba00df 2023-03-05 19:55:26,515 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-05 19:55:26,516 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1678046125369 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-05 19:55:26,516 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] snapshot.SnapshotManager(404): Snapshoting '{ ss=snaptb-1678046125369 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-03-05 19:55:26,516 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2428): (#5) Sleeping: 1000ms while waiting for snapshot completion. 2023-03-05 19:55:26,814 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(484): Convert to Single Snapshot Manifest for snaptb-1678046125369 2023-03-05 19:55:26,816 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifestV1(126): No regions under directory:hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/.hbase-snapshot/.tmp/snaptb-1678046125369 2023-03-05 19:55:27,273 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotDescriptionUtils(404): Sentinel is done, just moving the snapshot from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/.hbase-snapshot/.tmp/snaptb-1678046125369 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/.hbase-snapshot/snaptb-1678046125369 2023-03-05 19:55:27,517 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-05 19:55:27,517 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1678046125369 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-05 19:55:27,517 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] snapshot.SnapshotManager(404): Snapshoting '{ ss=snaptb-1678046125369 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-03-05 19:55:27,518 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2428): (#6) Sleeping: 2000ms while waiting for snapshot completion. 2023-03-05 19:55:28,088 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(229): Snapshot snaptb-1678046125369 of table testCompactionWithHFileLink completed 2023-03-05 19:55:28,088 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(246): Launching cleanup of working dir:hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/.hbase-snapshot/.tmp/snaptb-1678046125369 2023-03-05 19:55:28,088 ERROR [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(251): Couldn't delete snapshot working directory:hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/.hbase-snapshot/.tmp/snaptb-1678046125369 2023-03-05 19:55:28,088 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(257): Table snapshot journal : Running FLUSH table snapshot snaptb-1678046125369 C_M_SNAPSHOT_TABLE on table testCompactionWithHFileLink at 1678046125403Completed referencing HFiles for the mob region of table: testCompactionWithHFileLink at 1678046126814 (+1411 ms)Consolidate snapshot: snaptb-1678046125369 at 1678046126814Loading Region manifests for snaptb-1678046125369 at 1678046126814Writing data manifest for snaptb-1678046125369 at 1678046126832 (+18 ms)Verifying snapshot: snaptb-1678046125369 at 1678046127243 (+411 ms)Snapshot snaptb-1678046125369 of table testCompactionWithHFileLink completed at 1678046128088 (+845 ms) 2023-03-05 19:55:28,089 DEBUG [PEWorker-1] locking.LockProcedure(242): UNLOCKED pid=87, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED 2023-03-05 19:55:28,091 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=87, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED in 2.6850 sec 2023-03-05 19:55:29,518 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-05 19:55:29,519 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1678046125369 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-05 19:55:29,519 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] snapshot.SnapshotManager(401): Snapshot '{ ss=snaptb-1678046125369 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' has completed, notifying client. 2023-03-05 19:55:29,546 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-05 19:55:29,548 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(143): Procedure testCompactionWithHFileLink was in running list but was completed. Accepting new attempt. 2023-03-05 19:55:29,548 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-03-05 19:55:29,548 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-03-05 19:55:29,548 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:55:29,548 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-03-05 19:55:29,548 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-03-05 19:55:29,548 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-05 19:55:29,548 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-05 19:55:29,550 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:55:29,550 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:55:29,550 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:29,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:55:29,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-05 19:55:29,550 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:29,550 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-05 19:55:29,550 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-05 19:55:29,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-05 19:55:29,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-05 19:55:29,551 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-03-05 19:55:29,551 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-03-05 19:55:29,551 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-03-05 19:55:29,551 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:55:29,551 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-03-05 19:55:29,551 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1678046116082.942afd2d0a4ceef025fd5a607956d53d. 2023-03-05 19:55:29,551 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1678046116082.25af79a83c9b248c956b57fc8507ead6. 2023-03-05 19:55:29,551 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1678046116082.942afd2d0a4ceef025fd5a607956d53d. started... 2023-03-05 19:55:29,551 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1678046116082.25af79a83c9b248c956b57fc8507ead6. started... 2023-03-05 19:55:29,552 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 942afd2d0a4ceef025fd5a607956d53d 2/2 column families, dataSize=138 B heapSize=944 B 2023-03-05 19:55:29,552 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 25af79a83c9b248c956b57fc8507ead6 2/2 column families, dataSize=138 B heapSize=944 B 2023-03-05 19:55:29,556 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-05 19:55:29,556 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-05 19:55:29,556 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1678046116082.daa57ebba7c140c319a2f6facc725e4a. 2023-03-05 19:55:29,556 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1678046116082.daa57ebba7c140c319a2f6facc725e4a. started... 2023-03-05 19:55:29,556 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing daa57ebba7c140c319a2f6facc725e4a 2/2 column families, dataSize=138 B heapSize=944 B 2023-03-05 19:55:29,978 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/.tmp/family1/3f63328ddf3c467d9fd39f3c33b6fc37 2023-03-05 19:55:29,981 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 3f63328ddf3c467d9fd39f3c33b6fc37 2023-03-05 19:55:30,377 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/.tmp/family1/919251f88e944758ab162b6e01fda9cb 2023-03-05 19:55:30,378 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/.tmp/family1/892a5ccd7dc44aeb88d8c624c148896c 2023-03-05 19:55:30,381 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 919251f88e944758ab162b6e01fda9cb 2023-03-05 19:55:30,381 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 892a5ccd7dc44aeb88d8c624c148896c 2023-03-05 19:55:30,805 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/.tmp/family2/5d8fc361501b400bb17a913215f3e5ad 2023-03-05 19:55:30,809 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5d8fc361501b400bb17a913215f3e5ad 2023-03-05 19:55:30,809 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/.tmp/family1/3f63328ddf3c467d9fd39f3c33b6fc37 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family1/3f63328ddf3c467d9fd39f3c33b6fc37 2023-03-05 19:55:30,812 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 3f63328ddf3c467d9fd39f3c33b6fc37 2023-03-05 19:55:30,812 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family1/3f63328ddf3c467d9fd39f3c33b6fc37, entries=3, sequenceid=24, filesize=5.0 K 2023-03-05 19:55:30,813 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/.tmp/family2/5d8fc361501b400bb17a913215f3e5ad as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family2/5d8fc361501b400bb17a913215f3e5ad 2023-03-05 19:55:30,816 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5d8fc361501b400bb17a913215f3e5ad 2023-03-05 19:55:30,816 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family2/5d8fc361501b400bb17a913215f3e5ad, entries=1, sequenceid=24, filesize=5.0 K 2023-03-05 19:55:30,817 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for daa57ebba7c140c319a2f6facc725e4a in 1261ms, sequenceid=24, compaction requested=true 2023-03-05 19:55:30,817 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for daa57ebba7c140c319a2f6facc725e4a: 2023-03-05 19:55:30,817 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1678046116082.daa57ebba7c140c319a2f6facc725e4a. 2023-03-05 19:55:30,817 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-05 19:55:31,203 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/.tmp/family2/0101c448c42a4b6bae722f205e0ef131 2023-03-05 19:55:31,203 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/.tmp/family2/7ec80bea99654a38b0437a4486d66188 2023-03-05 19:55:31,207 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7ec80bea99654a38b0437a4486d66188 2023-03-05 19:55:31,207 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0101c448c42a4b6bae722f205e0ef131 2023-03-05 19:55:31,208 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/.tmp/family1/919251f88e944758ab162b6e01fda9cb as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family1/919251f88e944758ab162b6e01fda9cb 2023-03-05 19:55:31,208 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/.tmp/family1/892a5ccd7dc44aeb88d8c624c148896c as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family1/892a5ccd7dc44aeb88d8c624c148896c 2023-03-05 19:55:31,211 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 919251f88e944758ab162b6e01fda9cb 2023-03-05 19:55:31,211 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family1/919251f88e944758ab162b6e01fda9cb, entries=3, sequenceid=24, filesize=5.0 K 2023-03-05 19:55:31,211 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 892a5ccd7dc44aeb88d8c624c148896c 2023-03-05 19:55:31,211 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family1/892a5ccd7dc44aeb88d8c624c148896c, entries=3, sequenceid=24, filesize=5.0 K 2023-03-05 19:55:31,212 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/.tmp/family2/7ec80bea99654a38b0437a4486d66188 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family2/7ec80bea99654a38b0437a4486d66188 2023-03-05 19:55:31,212 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/.tmp/family2/0101c448c42a4b6bae722f205e0ef131 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family2/0101c448c42a4b6bae722f205e0ef131 2023-03-05 19:55:31,215 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7ec80bea99654a38b0437a4486d66188 2023-03-05 19:55:31,215 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family2/7ec80bea99654a38b0437a4486d66188, entries=1, sequenceid=24, filesize=5.0 K 2023-03-05 19:55:31,215 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0101c448c42a4b6bae722f205e0ef131 2023-03-05 19:55:31,216 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family2/0101c448c42a4b6bae722f205e0ef131, entries=1, sequenceid=24, filesize=5.0 K 2023-03-05 19:55:31,216 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 942afd2d0a4ceef025fd5a607956d53d in 1664ms, sequenceid=24, compaction requested=true 2023-03-05 19:55:31,216 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 942afd2d0a4ceef025fd5a607956d53d: 2023-03-05 19:55:31,216 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1678046116082.942afd2d0a4ceef025fd5a607956d53d. 2023-03-05 19:55:31,216 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-05 19:55:31,216 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 25af79a83c9b248c956b57fc8507ead6 in 1664ms, sequenceid=24, compaction requested=true 2023-03-05 19:55:31,216 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 25af79a83c9b248c956b57fc8507ead6: 2023-03-05 19:55:31,217 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1678046116082.25af79a83c9b248c956b57fc8507ead6. 2023-03-05 19:55:31,217 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-05 19:55:31,217 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-05 19:55:31,217 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:31,217 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-03-05 19:55:31,217 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-03-05 19:55:31,218 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-05 19:55:31,218 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:31,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:31,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:55:31,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:55:31,218 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-05 19:55:31,218 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-05 19:55:31,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:55:31,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:55:31,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-05 19:55:31,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:31,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:55:31,220 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-03-05 19:55:31,220 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-03-05 19:55:31,220 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1d97ab19[Count = 0] remaining members to acquire global barrier 2023-03-05 19:55:31,220 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-05 19:55:31,224 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-05 19:55:31,224 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-05 19:55:31,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-05 19:55:31,224 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-03-05 19:55:31,224 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-03-05 19:55:31,224 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,37037,1678046018983' in zk 2023-03-05 19:55:31,225 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:31,225 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-05 19:55:31,226 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:31,226 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-03-05 19:55:31,226 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:55:31,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:31,226 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-03-05 19:55:31,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:55:31,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:55:31,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:55:31,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:55:31,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-05 19:55:31,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:31,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:55:31,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-05 19:55:31,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:31,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,37037,1678046018983': 2023-03-05 19:55:31,228 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-03-05 19:55:31,228 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-03-05 19:55:31,228 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-05 19:55:31,228 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-05 19:55:31,228 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-03-05 19:55:31,228 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-05 19:55:31,229 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-05 19:55:31,229 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-05 19:55:31,229 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-05 19:55:31,229 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:55:31,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-05 19:55:31,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-05 19:55:31,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:55:31,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:55:31,230 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:55:31,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:55:31,230 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:31,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:55:31,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-05 19:55:31,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-05 19:55:31,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:55:31,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-05 19:55:31,231 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:31,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:31,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:55:31,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-05 19:55:31,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:31,233 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:55:31,233 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:31,233 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:55:31,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:55:31,233 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:55:31,233 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-03-05 19:55:31,233 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-05 19:55:31,233 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-05 19:55:31,233 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:55:31,233 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:55:31,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:55:31,233 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-05 19:55:31,233 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-03-05 19:55:31,233 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:31,233 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-05 19:55:31,233 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink because node does not exist (not an error) 2023-03-05 19:55:31,233 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-05 19:55:31,233 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-05 19:55:31,234 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-05 19:55:31,334 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-05 19:55:31,334 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-05 19:55:31,335 DEBUG [Listener at localhost/40787] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-05 19:55:31,335 DEBUG [Listener at localhost/40787] regionserver.HStore(1912): 942afd2d0a4ceef025fd5a607956d53d/family1 is initiating major compaction (all files) 2023-03-05 19:55:31,335 INFO [Listener at localhost/40787] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-05 19:55:31,336 INFO [Listener at localhost/40787] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-05 19:55:31,336 INFO [Listener at localhost/40787] regionserver.HRegion(2259): Starting compaction of 942afd2d0a4ceef025fd5a607956d53d/family1 in testCompactionWithHFileLink,1,1678046116082.942afd2d0a4ceef025fd5a607956d53d. 2023-03-05 19:55:31,336 INFO [Listener at localhost/40787] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family1/2ddb935fe9dc46778a4c450ff10d3c44, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family1/dc8108fdcc6a4a0a9720b3e2336cb620, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family1/c88f1bd396f4409ca38794d504f5c156, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family1/8139f54670e14aa3a572f9bdb2477108, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family1/919251f88e944758ab162b6e01fda9cb] into tmpdir=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/.tmp, totalSize=26.7 K 2023-03-05 19:55:31,336 DEBUG [Listener at localhost/40787] compactions.Compactor(207): Compacting 2ddb935fe9dc46778a4c450ff10d3c44, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678046118197 2023-03-05 19:55:31,337 DEBUG [Listener at localhost/40787] compactions.Compactor(207): Compacting dc8108fdcc6a4a0a9720b3e2336cb620, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678046119974 2023-03-05 19:55:31,337 DEBUG [Listener at localhost/40787] compactions.Compactor(207): Compacting c88f1bd396f4409ca38794d504f5c156, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678046121773 2023-03-05 19:55:31,337 DEBUG [Listener at localhost/40787] compactions.Compactor(207): Compacting 8139f54670e14aa3a572f9bdb2477108, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678046123595 2023-03-05 19:55:31,338 DEBUG [Listener at localhost/40787] compactions.Compactor(207): Compacting 919251f88e944758ab162b6e01fda9cb, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-03-05 19:55:31,349 INFO [Listener at localhost/40787] throttle.PressureAwareThroughputController(145): 942afd2d0a4ceef025fd5a607956d53d#family1#compaction#107 average throughput is 0.57 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-05 19:55:31,757 INFO [Listener at localhost/40787] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20230305859844562c574340a9dc77e79ba2e662_del 2023-03-05 19:55:31,757 INFO [Listener at localhost/40787] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230305859844562c574340a9dc77e79ba2e662_del to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230305859844562c574340a9dc77e79ba2e662_del 2023-03-05 19:55:32,569 INFO [Listener at localhost/40787] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ec9c98101b6045a7ae5e03e71336653c 2023-03-05 19:55:32,570 DEBUG [Listener at localhost/40787] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/.tmp/family1/ec9c98101b6045a7ae5e03e71336653c as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family1/ec9c98101b6045a7ae5e03e71336653c 2023-03-05 19:55:32,573 INFO [Listener at localhost/40787] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ec9c98101b6045a7ae5e03e71336653c 2023-03-05 19:55:32,574 INFO [Listener at localhost/40787] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 942afd2d0a4ceef025fd5a607956d53d/family1 of 942afd2d0a4ceef025fd5a607956d53d into ec9c98101b6045a7ae5e03e71336653c(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-05 19:55:32,574 DEBUG [Listener at localhost/40787] regionserver.HRegion(2289): Compaction status journal for 942afd2d0a4ceef025fd5a607956d53d: 2023-03-05 19:55:32,574 DEBUG [Listener at localhost/40787] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-05 19:55:32,574 DEBUG [Listener at localhost/40787] regionserver.HStore(1912): 942afd2d0a4ceef025fd5a607956d53d/family2 is initiating major compaction (all files) 2023-03-05 19:55:32,575 INFO [Listener at localhost/40787] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-05 19:55:32,575 INFO [Listener at localhost/40787] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-05 19:55:32,575 INFO [Listener at localhost/40787] regionserver.HRegion(2259): Starting compaction of 942afd2d0a4ceef025fd5a607956d53d/family2 in testCompactionWithHFileLink,1,1678046116082.942afd2d0a4ceef025fd5a607956d53d. 2023-03-05 19:55:32,575 INFO [Listener at localhost/40787] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family2/9a13755a01ce4e01b7e321e95b43b8cc, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family2/af93404d81154179a4c27fce0e335552, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family2/2c3de0e4dce5444ca51da5f66eaf06a1, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family2/55d239cb7531440088e28c0c9e6143d3, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family2/7ec80bea99654a38b0437a4486d66188] into tmpdir=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/.tmp, totalSize=25.4 K 2023-03-05 19:55:32,576 DEBUG [Listener at localhost/40787] compactions.Compactor(207): Compacting 9a13755a01ce4e01b7e321e95b43b8cc, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678046118197 2023-03-05 19:55:32,576 DEBUG [Listener at localhost/40787] compactions.Compactor(207): Compacting af93404d81154179a4c27fce0e335552, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678046119974 2023-03-05 19:55:32,576 DEBUG [Listener at localhost/40787] compactions.Compactor(207): Compacting 2c3de0e4dce5444ca51da5f66eaf06a1, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678046121773 2023-03-05 19:55:32,577 DEBUG [Listener at localhost/40787] compactions.Compactor(207): Compacting 55d239cb7531440088e28c0c9e6143d3, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678046123595 2023-03-05 19:55:32,577 DEBUG [Listener at localhost/40787] compactions.Compactor(207): Compacting 7ec80bea99654a38b0437a4486d66188, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-03-05 19:55:32,580 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family1/2ddb935fe9dc46778a4c450ff10d3c44, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family1/dc8108fdcc6a4a0a9720b3e2336cb620, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family1/c88f1bd396f4409ca38794d504f5c156, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family1/8139f54670e14aa3a572f9bdb2477108, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family1/919251f88e944758ab162b6e01fda9cb] to archive 2023-03-05 19:55:32,581 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-05 19:55:32,582 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family1/2ddb935fe9dc46778a4c450ff10d3c44 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family1/2ddb935fe9dc46778a4c450ff10d3c44 2023-03-05 19:55:32,583 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family1/dc8108fdcc6a4a0a9720b3e2336cb620 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family1/dc8108fdcc6a4a0a9720b3e2336cb620 2023-03-05 19:55:32,585 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family1/c88f1bd396f4409ca38794d504f5c156 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family1/c88f1bd396f4409ca38794d504f5c156 2023-03-05 19:55:32,586 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family1/8139f54670e14aa3a572f9bdb2477108 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family1/8139f54670e14aa3a572f9bdb2477108 2023-03-05 19:55:32,587 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family1/919251f88e944758ab162b6e01fda9cb to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family1/919251f88e944758ab162b6e01fda9cb 2023-03-05 19:55:32,588 INFO [Listener at localhost/40787] throttle.PressureAwareThroughputController(145): 942afd2d0a4ceef025fd5a607956d53d#family2#compaction#108 average throughput is 0.35 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-05 19:55:32,995 INFO [Listener at localhost/40787] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20230305a1ce4fce5ac245639bc446818ed6cf66_del 2023-03-05 19:55:32,996 INFO [Listener at localhost/40787] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230305a1ce4fce5ac245639bc446818ed6cf66_del to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230305a1ce4fce5ac245639bc446818ed6cf66_del 2023-03-05 19:55:33,808 INFO [Listener at localhost/40787] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e2d807631cc1463ab28bd4f9dd62e867 2023-03-05 19:55:33,809 DEBUG [Listener at localhost/40787] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/.tmp/family2/e2d807631cc1463ab28bd4f9dd62e867 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family2/e2d807631cc1463ab28bd4f9dd62e867 2023-03-05 19:55:33,812 INFO [Listener at localhost/40787] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e2d807631cc1463ab28bd4f9dd62e867 2023-03-05 19:55:33,813 INFO [Listener at localhost/40787] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 942afd2d0a4ceef025fd5a607956d53d/family2 of 942afd2d0a4ceef025fd5a607956d53d into e2d807631cc1463ab28bd4f9dd62e867(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-05 19:55:33,813 DEBUG [Listener at localhost/40787] regionserver.HRegion(2289): Compaction status journal for 942afd2d0a4ceef025fd5a607956d53d: 2023-03-05 19:55:33,813 DEBUG [Listener at localhost/40787] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-05 19:55:33,813 DEBUG [Listener at localhost/40787] regionserver.HStore(1912): 25af79a83c9b248c956b57fc8507ead6/family1 is initiating major compaction (all files) 2023-03-05 19:55:33,814 INFO [Listener at localhost/40787] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-05 19:55:33,814 INFO [Listener at localhost/40787] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-05 19:55:33,814 INFO [Listener at localhost/40787] regionserver.HRegion(2259): Starting compaction of 25af79a83c9b248c956b57fc8507ead6/family1 in testCompactionWithHFileLink,2,1678046116082.25af79a83c9b248c956b57fc8507ead6. 2023-03-05 19:55:33,814 INFO [Listener at localhost/40787] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family1/9a9155a47d074f5f9005ddf6337e173e, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family1/24f57a619907417b8e8a0222c800bc07, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family1/3004199e82364ee9abaf18f0f843fc76, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family1/6b8608e70a864d6e8a4e55c85261adfe, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family1/892a5ccd7dc44aeb88d8c624c148896c] into tmpdir=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/.tmp, totalSize=26.7 K 2023-03-05 19:55:33,814 DEBUG [Listener at localhost/40787] compactions.Compactor(207): Compacting 9a9155a47d074f5f9005ddf6337e173e, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678046118198 2023-03-05 19:55:33,814 DEBUG [Listener at localhost/40787] compactions.Compactor(207): Compacting 24f57a619907417b8e8a0222c800bc07, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678046119975 2023-03-05 19:55:33,815 DEBUG [Listener at localhost/40787] compactions.Compactor(207): Compacting 3004199e82364ee9abaf18f0f843fc76, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678046121774 2023-03-05 19:55:33,815 DEBUG [Listener at localhost/40787] compactions.Compactor(207): Compacting 6b8608e70a864d6e8a4e55c85261adfe, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678046123595 2023-03-05 19:55:33,815 DEBUG [Listener at localhost/40787] compactions.Compactor(207): Compacting 892a5ccd7dc44aeb88d8c624c148896c, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-03-05 19:55:33,825 INFO [Listener at localhost/40787] throttle.PressureAwareThroughputController(145): 25af79a83c9b248c956b57fc8507ead6#family1#compaction#109 average throughput is 0.57 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-05 19:55:33,881 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family2/9a13755a01ce4e01b7e321e95b43b8cc, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family2/af93404d81154179a4c27fce0e335552, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family2/2c3de0e4dce5444ca51da5f66eaf06a1, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family2/55d239cb7531440088e28c0c9e6143d3, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family2/7ec80bea99654a38b0437a4486d66188] to archive 2023-03-05 19:55:33,882 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-05 19:55:33,883 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family2/9a13755a01ce4e01b7e321e95b43b8cc to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family2/9a13755a01ce4e01b7e321e95b43b8cc 2023-03-05 19:55:33,885 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family2/af93404d81154179a4c27fce0e335552 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family2/af93404d81154179a4c27fce0e335552 2023-03-05 19:55:33,886 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family2/2c3de0e4dce5444ca51da5f66eaf06a1 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family2/2c3de0e4dce5444ca51da5f66eaf06a1 2023-03-05 19:55:33,887 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family2/55d239cb7531440088e28c0c9e6143d3 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family2/55d239cb7531440088e28c0c9e6143d3 2023-03-05 19:55:33,888 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family2/7ec80bea99654a38b0437a4486d66188 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family2/7ec80bea99654a38b0437a4486d66188 2023-03-05 19:55:34,242 INFO [Listener at localhost/40787] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c2023030594dac2d83eea492f86f773d941f88e34_del 2023-03-05 19:55:34,242 INFO [Listener at localhost/40787] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023030594dac2d83eea492f86f773d941f88e34_del to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023030594dac2d83eea492f86f773d941f88e34_del 2023-03-05 19:55:35,055 INFO [Listener at localhost/40787] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0a08f6f51c1546e184b4393a89afb484 2023-03-05 19:55:35,056 DEBUG [Listener at localhost/40787] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/.tmp/family1/0a08f6f51c1546e184b4393a89afb484 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family1/0a08f6f51c1546e184b4393a89afb484 2023-03-05 19:55:35,059 INFO [Listener at localhost/40787] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0a08f6f51c1546e184b4393a89afb484 2023-03-05 19:55:35,060 INFO [Listener at localhost/40787] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 25af79a83c9b248c956b57fc8507ead6/family1 of 25af79a83c9b248c956b57fc8507ead6 into 0a08f6f51c1546e184b4393a89afb484(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-05 19:55:35,060 DEBUG [Listener at localhost/40787] regionserver.HRegion(2289): Compaction status journal for 25af79a83c9b248c956b57fc8507ead6: 2023-03-05 19:55:35,060 DEBUG [Listener at localhost/40787] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-05 19:55:35,060 DEBUG [Listener at localhost/40787] regionserver.HStore(1912): 25af79a83c9b248c956b57fc8507ead6/family2 is initiating major compaction (all files) 2023-03-05 19:55:35,060 INFO [Listener at localhost/40787] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-05 19:55:35,060 INFO [Listener at localhost/40787] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-05 19:55:35,060 INFO [Listener at localhost/40787] regionserver.HRegion(2259): Starting compaction of 25af79a83c9b248c956b57fc8507ead6/family2 in testCompactionWithHFileLink,2,1678046116082.25af79a83c9b248c956b57fc8507ead6. 2023-03-05 19:55:35,061 INFO [Listener at localhost/40787] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family2/22cbfc0fa2af442f976458cd740e398d, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family2/e2b047fb6d65462d939ae96ac7375619, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family2/1dfce094a6164169b84ae1182f0260e1, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family2/11fd411913d44c5187ed3f273ed56bdd, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family2/0101c448c42a4b6bae722f205e0ef131] into tmpdir=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/.tmp, totalSize=25.4 K 2023-03-05 19:55:35,061 DEBUG [Listener at localhost/40787] compactions.Compactor(207): Compacting 22cbfc0fa2af442f976458cd740e398d, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678046118198 2023-03-05 19:55:35,061 DEBUG [Listener at localhost/40787] compactions.Compactor(207): Compacting e2b047fb6d65462d939ae96ac7375619, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678046119975 2023-03-05 19:55:35,062 DEBUG [Listener at localhost/40787] compactions.Compactor(207): Compacting 1dfce094a6164169b84ae1182f0260e1, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678046121774 2023-03-05 19:55:35,062 DEBUG [Listener at localhost/40787] compactions.Compactor(207): Compacting 11fd411913d44c5187ed3f273ed56bdd, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678046123595 2023-03-05 19:55:35,062 DEBUG [Listener at localhost/40787] compactions.Compactor(207): Compacting 0101c448c42a4b6bae722f205e0ef131, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-03-05 19:55:35,072 INFO [Listener at localhost/40787] throttle.PressureAwareThroughputController(145): 25af79a83c9b248c956b57fc8507ead6#family2#compaction#110 average throughput is 0.53 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-05 19:55:35,081 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family1/9a9155a47d074f5f9005ddf6337e173e, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family1/24f57a619907417b8e8a0222c800bc07, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family1/3004199e82364ee9abaf18f0f843fc76, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family1/6b8608e70a864d6e8a4e55c85261adfe, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family1/892a5ccd7dc44aeb88d8c624c148896c] to archive 2023-03-05 19:55:35,081 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-05 19:55:35,083 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family1/9a9155a47d074f5f9005ddf6337e173e to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family1/9a9155a47d074f5f9005ddf6337e173e 2023-03-05 19:55:35,084 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family1/24f57a619907417b8e8a0222c800bc07 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family1/24f57a619907417b8e8a0222c800bc07 2023-03-05 19:55:35,084 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family1/3004199e82364ee9abaf18f0f843fc76 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family1/3004199e82364ee9abaf18f0f843fc76 2023-03-05 19:55:35,085 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family1/6b8608e70a864d6e8a4e55c85261adfe to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family1/6b8608e70a864d6e8a4e55c85261adfe 2023-03-05 19:55:35,086 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family1/892a5ccd7dc44aeb88d8c624c148896c to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family1/892a5ccd7dc44aeb88d8c624c148896c 2023-03-05 19:55:35,481 INFO [Listener at localhost/40787] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230305b01248d686924c22a35fa5280ae31538_del 2023-03-05 19:55:35,481 INFO [Listener at localhost/40787] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230305b01248d686924c22a35fa5280ae31538_del to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230305b01248d686924c22a35fa5280ae31538_del 2023-03-05 19:55:36,294 INFO [Listener at localhost/40787] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8233b8c3ed9a4588916d13d67112fe6e 2023-03-05 19:55:36,294 DEBUG [Listener at localhost/40787] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/.tmp/family2/8233b8c3ed9a4588916d13d67112fe6e as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family2/8233b8c3ed9a4588916d13d67112fe6e 2023-03-05 19:55:36,298 INFO [Listener at localhost/40787] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8233b8c3ed9a4588916d13d67112fe6e 2023-03-05 19:55:36,299 INFO [Listener at localhost/40787] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 25af79a83c9b248c956b57fc8507ead6/family2 of 25af79a83c9b248c956b57fc8507ead6 into 8233b8c3ed9a4588916d13d67112fe6e(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-05 19:55:36,299 DEBUG [Listener at localhost/40787] regionserver.HRegion(2289): Compaction status journal for 25af79a83c9b248c956b57fc8507ead6: 2023-03-05 19:55:36,299 DEBUG [Listener at localhost/40787] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-05 19:55:36,299 DEBUG [Listener at localhost/40787] regionserver.HStore(1912): daa57ebba7c140c319a2f6facc725e4a/family1 is initiating major compaction (all files) 2023-03-05 19:55:36,300 INFO [Listener at localhost/40787] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-05 19:55:36,300 INFO [Listener at localhost/40787] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-05 19:55:36,300 INFO [Listener at localhost/40787] regionserver.HRegion(2259): Starting compaction of daa57ebba7c140c319a2f6facc725e4a/family1 in testCompactionWithHFileLink,,1678046116082.daa57ebba7c140c319a2f6facc725e4a. 2023-03-05 19:55:36,300 INFO [Listener at localhost/40787] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family1/cdcb9dd831794696af2aaf30754840ee, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family1/f6b69932b5da4e0d9155ef905eacf40c, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family1/43db0ea3bd404e2fa10949ad9df289b4, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family1/e02bc7cebc7e44af831019cee45cd29b, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family1/3f63328ddf3c467d9fd39f3c33b6fc37] into tmpdir=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/.tmp, totalSize=26.7 K 2023-03-05 19:55:36,300 DEBUG [Listener at localhost/40787] compactions.Compactor(207): Compacting cdcb9dd831794696af2aaf30754840ee, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678046118195 2023-03-05 19:55:36,301 DEBUG [Listener at localhost/40787] compactions.Compactor(207): Compacting f6b69932b5da4e0d9155ef905eacf40c, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678046119974 2023-03-05 19:55:36,301 DEBUG [Listener at localhost/40787] compactions.Compactor(207): Compacting 43db0ea3bd404e2fa10949ad9df289b4, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678046121773 2023-03-05 19:55:36,301 DEBUG [Listener at localhost/40787] compactions.Compactor(207): Compacting e02bc7cebc7e44af831019cee45cd29b, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678046123595 2023-03-05 19:55:36,301 DEBUG [Listener at localhost/40787] compactions.Compactor(207): Compacting 3f63328ddf3c467d9fd39f3c33b6fc37, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-03-05 19:55:36,312 INFO [Listener at localhost/40787] throttle.PressureAwareThroughputController(145): daa57ebba7c140c319a2f6facc725e4a#family1#compaction#111 average throughput is 0.85 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-05 19:55:36,380 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family2/22cbfc0fa2af442f976458cd740e398d, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family2/e2b047fb6d65462d939ae96ac7375619, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family2/1dfce094a6164169b84ae1182f0260e1, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family2/11fd411913d44c5187ed3f273ed56bdd, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family2/0101c448c42a4b6bae722f205e0ef131] to archive 2023-03-05 19:55:36,381 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-05 19:55:36,382 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family2/22cbfc0fa2af442f976458cd740e398d to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family2/22cbfc0fa2af442f976458cd740e398d 2023-03-05 19:55:36,383 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family2/e2b047fb6d65462d939ae96ac7375619 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family2/e2b047fb6d65462d939ae96ac7375619 2023-03-05 19:55:36,384 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family2/1dfce094a6164169b84ae1182f0260e1 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family2/1dfce094a6164169b84ae1182f0260e1 2023-03-05 19:55:36,385 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family2/11fd411913d44c5187ed3f273ed56bdd to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family2/11fd411913d44c5187ed3f273ed56bdd 2023-03-05 19:55:36,386 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family2/0101c448c42a4b6bae722f205e0ef131 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family2/0101c448c42a4b6bae722f205e0ef131 2023-03-05 19:55:36,721 INFO [Listener at localhost/40787] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20230305742b208c36f540e5b62794234602a315_del 2023-03-05 19:55:36,721 INFO [Listener at localhost/40787] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230305742b208c36f540e5b62794234602a315_del to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230305742b208c36f540e5b62794234602a315_del 2023-03-05 19:55:37,535 INFO [Listener at localhost/40787] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 531e515ba5ab40f0913a9ad406709567 2023-03-05 19:55:37,535 DEBUG [Listener at localhost/40787] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/.tmp/family1/531e515ba5ab40f0913a9ad406709567 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family1/531e515ba5ab40f0913a9ad406709567 2023-03-05 19:55:37,539 INFO [Listener at localhost/40787] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 531e515ba5ab40f0913a9ad406709567 2023-03-05 19:55:37,540 INFO [Listener at localhost/40787] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in daa57ebba7c140c319a2f6facc725e4a/family1 of daa57ebba7c140c319a2f6facc725e4a into 531e515ba5ab40f0913a9ad406709567(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-05 19:55:37,540 DEBUG [Listener at localhost/40787] regionserver.HRegion(2289): Compaction status journal for daa57ebba7c140c319a2f6facc725e4a: 2023-03-05 19:55:37,540 DEBUG [Listener at localhost/40787] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-05 19:55:37,540 DEBUG [Listener at localhost/40787] regionserver.HStore(1912): daa57ebba7c140c319a2f6facc725e4a/family2 is initiating major compaction (all files) 2023-03-05 19:55:37,540 INFO [Listener at localhost/40787] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-05 19:55:37,540 INFO [Listener at localhost/40787] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-05 19:55:37,540 INFO [Listener at localhost/40787] regionserver.HRegion(2259): Starting compaction of daa57ebba7c140c319a2f6facc725e4a/family2 in testCompactionWithHFileLink,,1678046116082.daa57ebba7c140c319a2f6facc725e4a. 2023-03-05 19:55:37,541 INFO [Listener at localhost/40787] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family2/c42309b45f3243f2a021144945ad30f4, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family2/c2d4bdd0af534bf8b270644df2f98eaf, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family2/13ca0708b05643899307a03f0fcc261a, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family2/9173733bfa77411fa7038a412b2f97e1, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family2/5d8fc361501b400bb17a913215f3e5ad] into tmpdir=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/.tmp, totalSize=25.4 K 2023-03-05 19:55:37,541 DEBUG [Listener at localhost/40787] compactions.Compactor(207): Compacting c42309b45f3243f2a021144945ad30f4, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678046118195 2023-03-05 19:55:37,541 DEBUG [Listener at localhost/40787] compactions.Compactor(207): Compacting c2d4bdd0af534bf8b270644df2f98eaf, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678046119974 2023-03-05 19:55:37,541 DEBUG [Listener at localhost/40787] compactions.Compactor(207): Compacting 13ca0708b05643899307a03f0fcc261a, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678046121773 2023-03-05 19:55:37,542 DEBUG [Listener at localhost/40787] compactions.Compactor(207): Compacting 9173733bfa77411fa7038a412b2f97e1, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678046123595 2023-03-05 19:55:37,542 DEBUG [Listener at localhost/40787] compactions.Compactor(207): Compacting 5d8fc361501b400bb17a913215f3e5ad, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-03-05 19:55:37,553 INFO [Listener at localhost/40787] throttle.PressureAwareThroughputController(145): daa57ebba7c140c319a2f6facc725e4a#family2#compaction#112 average throughput is 0.53 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-05 19:55:37,582 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family1/cdcb9dd831794696af2aaf30754840ee, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family1/f6b69932b5da4e0d9155ef905eacf40c, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family1/43db0ea3bd404e2fa10949ad9df289b4, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family1/e02bc7cebc7e44af831019cee45cd29b, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family1/3f63328ddf3c467d9fd39f3c33b6fc37] to archive 2023-03-05 19:55:37,582 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-05 19:55:37,584 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family1/cdcb9dd831794696af2aaf30754840ee to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family1/cdcb9dd831794696af2aaf30754840ee 2023-03-05 19:55:37,585 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family1/f6b69932b5da4e0d9155ef905eacf40c to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family1/f6b69932b5da4e0d9155ef905eacf40c 2023-03-05 19:55:37,585 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family1/43db0ea3bd404e2fa10949ad9df289b4 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family1/43db0ea3bd404e2fa10949ad9df289b4 2023-03-05 19:55:37,586 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family1/e02bc7cebc7e44af831019cee45cd29b to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family1/e02bc7cebc7e44af831019cee45cd29b 2023-03-05 19:55:37,587 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family1/3f63328ddf3c467d9fd39f3c33b6fc37 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family1/3f63328ddf3c467d9fd39f3c33b6fc37 2023-03-05 19:55:37,966 INFO [Listener at localhost/40787] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e202303050e8f89505ca74920aa760c9a68a9cea6_del 2023-03-05 19:55:37,966 INFO [Listener at localhost/40787] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303050e8f89505ca74920aa760c9a68a9cea6_del to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202303050e8f89505ca74920aa760c9a68a9cea6_del 2023-03-05 19:55:38,779 INFO [Listener at localhost/40787] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for bd91cfa3cac24d97bd240ffd35d114aa 2023-03-05 19:55:38,779 DEBUG [Listener at localhost/40787] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/.tmp/family2/bd91cfa3cac24d97bd240ffd35d114aa as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family2/bd91cfa3cac24d97bd240ffd35d114aa 2023-03-05 19:55:38,783 INFO [Listener at localhost/40787] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for bd91cfa3cac24d97bd240ffd35d114aa 2023-03-05 19:55:38,783 INFO [Listener at localhost/40787] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in daa57ebba7c140c319a2f6facc725e4a/family2 of daa57ebba7c140c319a2f6facc725e4a into bd91cfa3cac24d97bd240ffd35d114aa(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-05 19:55:38,784 DEBUG [Listener at localhost/40787] regionserver.HRegion(2289): Compaction status journal for daa57ebba7c140c319a2f6facc725e4a: 2023-03-05 19:55:38,789 DEBUG [Listener at localhost/40787] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-05 19:55:38,792 DEBUG [Listener at localhost/40787] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-05 19:55:38,795 DEBUG [Listener at localhost/40787] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-05 19:55:38,796 DEBUG [Listener at localhost/40787] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-05 19:55:38,803 INFO [Listener at localhost/40787] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-03-05 19:55:38,835 INFO [Listener at localhost/40787] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 3 del files, 12 selected files, and 0 irrelevant files table 'testCompactionWithHFileLink' and column 'family1' 2023-03-05 19:55:38,838 INFO [Listener at localhost/40787] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20230305742b208c36f540e5b62794234602a315_del 2023-03-05 19:55:38,841 INFO [Listener at localhost/40787] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20230305859844562c574340a9dc77e79ba2e662_del 2023-03-05 19:55:38,843 INFO [Listener at localhost/40787] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c2023030594dac2d83eea492f86f773d941f88e34_del 2023-03-05 19:55:38,843 INFO [Listener at localhost/40787] compactions.PartitionedMobCompactor(350): After merging, there are 3 del files. table='testCompactionWithHFileLink' column='family1' 2023-03-05 19:55:38,844 DEBUG [Listener at localhost/40787] zookeeper.ReadOnlyZKClient(139): Connect 0x375aed01 to 127.0.0.1:62957 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-05 19:55:38,849 DEBUG [Listener at localhost/40787] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@869b1f4, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-05 19:55:38,849 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230305 for table='testCompactionWithHFileLink' and column='family1' 2023-03-05 19:55:38,849 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230305 for table='testCompactionWithHFileLink' and column='family1' 2023-03-05 19:55:38,849 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c4ca4238a0b923820dcc509a6f75849b20230305 for table='testCompactionWithHFileLink' and column='family1' 2023-03-05 19:55:38,880 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family2/c42309b45f3243f2a021144945ad30f4, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family2/c2d4bdd0af534bf8b270644df2f98eaf, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family2/13ca0708b05643899307a03f0fcc261a, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family2/9173733bfa77411fa7038a412b2f97e1, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family2/5d8fc361501b400bb17a913215f3e5ad] to archive 2023-03-05 19:55:38,881 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-05 19:55:38,884 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family2/c42309b45f3243f2a021144945ad30f4 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family2/c42309b45f3243f2a021144945ad30f4 2023-03-05 19:55:38,886 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family2/c2d4bdd0af534bf8b270644df2f98eaf to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family2/c2d4bdd0af534bf8b270644df2f98eaf 2023-03-05 19:55:38,893 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family2/13ca0708b05643899307a03f0fcc261a to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family2/13ca0708b05643899307a03f0fcc261a 2023-03-05 19:55:38,896 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family2/9173733bfa77411fa7038a412b2f97e1 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family2/9173733bfa77411fa7038a412b2f97e1 2023-03-05 19:55:38,898 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family2/5d8fc361501b400bb17a913215f3e5ad to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family2/5d8fc361501b400bb17a913215f3e5ad 2023-03-05 19:55:39,307 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303053423757e17c645b0813f1ec26bbf3247 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202303053423757e17c645b0813f1ec26bbf3247 2023-03-05 19:55:39,307 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230305 table='testCompactionWithHFileLink' column='family1' 2023-03-05 19:55:39,320 DEBUG [MobFileCompactionChore-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-05 19:55:39,323 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:38242, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-05 19:55:39,329 DEBUG [MobFileCompactionChore-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-05 19:55:39,331 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:51714, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-05 19:55:39,338 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230305/family1/681726f9add342ee86f1d1d72152c80f first=Optional[0\x00\x00\x00\x01] last=Optional[0\x00\x00\x00\x07] 2023-03-05 19:55:39,339 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testCompactionWithHFileLink,,1678046116082.daa57ebba7c140c319a2f6facc725e4a., hostname=jenkins-hbase4.apache.org,37037,1678046018983, seqNum=2 for row with hfile group [{family1,hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230305/family1/681726f9add342ee86f1d1d72152c80f}] 2023-03-05 19:55:39,343 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(763): Validating hfile at hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230305/family1/681726f9add342ee86f1d1d72152c80f for inclusion in daa57ebba7c140c319a2f6facc725e4a/family1 2023-03-05 19:55:39,344 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(775): HFile bounds: first=0\x00\x00\x00\x01 last=0\x00\x00\x00\x07 2023-03-05 19:55:39,344 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(777): Region bounds: first= last=1 2023-03-05 19:55:39,345 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HRegion(2446): Flush status journal for daa57ebba7c140c319a2f6facc725e4a: 2023-03-05 19:55:39,345 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230305/family1/681726f9add342ee86f1d1d72152c80f to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testCompactionWithHFileLink__6knanmi3hrpd0tgi2kf4dlrntfjf8e1nk6617il5e433cd750tgq1sjpm8nve2ag/family1/681726f9add342ee86f1d1d72152c80f 2023-03-05 19:55:39,347 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testCompactionWithHFileLink__6knanmi3hrpd0tgi2kf4dlrntfjf8e1nk6617il5e433cd750tgq1sjpm8nve2ag/family1/681726f9add342ee86f1d1d72152c80f as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family1/301c6a15916c4e289e14b873cb6f18fc_SeqId_29_ 2023-03-05 19:55:39,347 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(849): Loaded HFile hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testCompactionWithHFileLink__6knanmi3hrpd0tgi2kf4dlrntfjf8e1nk6617il5e433cd750tgq1sjpm8nve2ag/family1/681726f9add342ee86f1d1d72152c80f into daa57ebba7c140c319a2f6facc725e4a/family1 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family1/301c6a15916c4e289e14b873cb6f18fc_SeqId_29_ - updating store file list. 2023-03-05 19:55:39,350 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(882): Loaded HFile hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family1/301c6a15916c4e289e14b873cb6f18fc_SeqId_29_ into daa57ebba7c140c319a2f6facc725e4a/family1 2023-03-05 19:55:39,350 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(855): Successfully loaded hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testCompactionWithHFileLink__6knanmi3hrpd0tgi2kf4dlrntfjf8e1nk6617il5e433cd750tgq1sjpm8nve2ag/family1/681726f9add342ee86f1d1d72152c80f into daa57ebba7c140c319a2f6facc725e4a/family1 (new location: hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family1/301c6a15916c4e289e14b873cb6f18fc_SeqId_29_) 2023-03-05 19:55:39,351 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testCompactionWithHFileLink__6knanmi3hrpd0tgi2kf4dlrntfjf8e1nk6617il5e433cd750tgq1sjpm8nve2ag/family1/681726f9add342ee86f1d1d72152c80f 2023-03-05 19:55:39,352 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230305 table='testCompactionWithHFileLink' column='family1' 2023-03-05 19:55:39,353 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-05 19:55:39,354 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202303056a09d4fe8b054b179c0313b89fbc21ce to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202303056a09d4fe8b054b179c0313b89fbc21ce 2023-03-05 19:55:39,355 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230305a33235cee5224842811abef82371aea6 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230305a33235cee5224842811abef82371aea6 2023-03-05 19:55:39,356 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230305da73569395114dad837fdbde0dcbba50 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230305da73569395114dad837fdbde0dcbba50 2023-03-05 19:55:39,356 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230305f43d564866564d9e939ce4a0e9ba6b2f to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230305f43d564866564d9e939ce4a0e9ba6b2f 2023-03-05 19:55:39,357 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-03-05 19:55:39,357 DEBUG [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230305' 2023-03-05 19:55:39,357 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230305 for table='testCompactionWithHFileLink' and column='family1' 2023-03-05 19:55:39,705 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230305ea913a4b69504c818e9b254877d6ee2b to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230305ea913a4b69504c818e9b254877d6ee2b 2023-03-05 19:55:39,706 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230305 table='testCompactionWithHFileLink' column='family1' 2023-03-05 19:55:39,708 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230305d8d40ebe0cf6478b93eb284f958283c5 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230305d8d40ebe0cf6478b93eb284f958283c5 2023-03-05 19:55:39,708 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230305 table='testCompactionWithHFileLink' column='family1' 2023-03-05 19:55:39,735 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230305/family1/4af89abac50a46daba2c4175180d74de first=Optional[1\x00\x00\x00\x01] last=Optional[1\x00\x00\x00\x07] 2023-03-05 19:55:39,737 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testCompactionWithHFileLink,1,1678046116082.942afd2d0a4ceef025fd5a607956d53d., hostname=jenkins-hbase4.apache.org,37037,1678046018983, seqNum=2 for row 1 with hfile group [{family1,hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230305/family1/4af89abac50a46daba2c4175180d74de}] 2023-03-05 19:55:39,738 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230305/family1/c4b1350024f44411968a7476048df950 first=Optional[2\x00\x00\x00\x01] last=Optional[2\x00\x00\x00\x07] 2023-03-05 19:55:39,738 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testCompactionWithHFileLink,2,1678046116082.25af79a83c9b248c956b57fc8507ead6., hostname=jenkins-hbase4.apache.org,37037,1678046018983, seqNum=2 for row 2 with hfile group [{family1,hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230305/family1/c4b1350024f44411968a7476048df950}] 2023-03-05 19:55:39,740 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(763): Validating hfile at hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230305/family1/4af89abac50a46daba2c4175180d74de for inclusion in 942afd2d0a4ceef025fd5a607956d53d/family1 2023-03-05 19:55:39,740 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.HStore(763): Validating hfile at hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230305/family1/c4b1350024f44411968a7476048df950 for inclusion in 25af79a83c9b248c956b57fc8507ead6/family1 2023-03-05 19:55:39,743 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(775): HFile bounds: first=1\x00\x00\x00\x01 last=1\x00\x00\x00\x07 2023-03-05 19:55:39,743 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(777): Region bounds: first=1 last=2 2023-03-05 19:55:39,743 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.HStore(775): HFile bounds: first=2\x00\x00\x00\x01 last=2\x00\x00\x00\x07 2023-03-05 19:55:39,743 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.HStore(777): Region bounds: first=2 last= 2023-03-05 19:55:39,743 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HRegion(2446): Flush status journal for 942afd2d0a4ceef025fd5a607956d53d: 2023-03-05 19:55:39,743 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.HRegion(2446): Flush status journal for 25af79a83c9b248c956b57fc8507ead6: 2023-03-05 19:55:39,744 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230305/family1/c4b1350024f44411968a7476048df950 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testCompactionWithHFileLink__pd3mv65fnc63j3m6gl00v0rd7gtboomvhs97qpdef95bsu81pkk6ng8e58fc122r/family1/c4b1350024f44411968a7476048df950 2023-03-05 19:55:39,746 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230305/family1/4af89abac50a46daba2c4175180d74de to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testCompactionWithHFileLink__5vpdqq4g5ifvg6b86havldm4nss3tu8cvjsoltrio3jnc772o3b91n1bahv3v2ib/family1/4af89abac50a46daba2c4175180d74de 2023-03-05 19:55:39,748 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testCompactionWithHFileLink__pd3mv65fnc63j3m6gl00v0rd7gtboomvhs97qpdef95bsu81pkk6ng8e58fc122r/family1/c4b1350024f44411968a7476048df950 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family1/b8fa05b9e21c43f2b863d96067141cb0_SeqId_29_ 2023-03-05 19:55:39,748 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.HStore(849): Loaded HFile hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testCompactionWithHFileLink__pd3mv65fnc63j3m6gl00v0rd7gtboomvhs97qpdef95bsu81pkk6ng8e58fc122r/family1/c4b1350024f44411968a7476048df950 into 25af79a83c9b248c956b57fc8507ead6/family1 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family1/b8fa05b9e21c43f2b863d96067141cb0_SeqId_29_ - updating store file list. 2023-03-05 19:55:39,749 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testCompactionWithHFileLink__5vpdqq4g5ifvg6b86havldm4nss3tu8cvjsoltrio3jnc772o3b91n1bahv3v2ib/family1/4af89abac50a46daba2c4175180d74de as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family1/ea8349daeb6649fea9912bed6e5461c7_SeqId_29_ 2023-03-05 19:55:39,749 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(849): Loaded HFile hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testCompactionWithHFileLink__5vpdqq4g5ifvg6b86havldm4nss3tu8cvjsoltrio3jnc772o3b91n1bahv3v2ib/family1/4af89abac50a46daba2c4175180d74de into 942afd2d0a4ceef025fd5a607956d53d/family1 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family1/ea8349daeb6649fea9912bed6e5461c7_SeqId_29_ - updating store file list. 2023-03-05 19:55:39,752 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.HStore(882): Loaded HFile hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family1/b8fa05b9e21c43f2b863d96067141cb0_SeqId_29_ into 25af79a83c9b248c956b57fc8507ead6/family1 2023-03-05 19:55:39,752 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.HStore(855): Successfully loaded hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testCompactionWithHFileLink__pd3mv65fnc63j3m6gl00v0rd7gtboomvhs97qpdef95bsu81pkk6ng8e58fc122r/family1/c4b1350024f44411968a7476048df950 into 25af79a83c9b248c956b57fc8507ead6/family1 (new location: hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family1/b8fa05b9e21c43f2b863d96067141cb0_SeqId_29_) 2023-03-05 19:55:39,752 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testCompactionWithHFileLink__pd3mv65fnc63j3m6gl00v0rd7gtboomvhs97qpdef95bsu81pkk6ng8e58fc122r/family1/c4b1350024f44411968a7476048df950 2023-03-05 19:55:39,752 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(882): Loaded HFile hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family1/ea8349daeb6649fea9912bed6e5461c7_SeqId_29_ into 942afd2d0a4ceef025fd5a607956d53d/family1 2023-03-05 19:55:39,753 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(855): Successfully loaded hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testCompactionWithHFileLink__5vpdqq4g5ifvg6b86havldm4nss3tu8cvjsoltrio3jnc772o3b91n1bahv3v2ib/family1/4af89abac50a46daba2c4175180d74de into 942afd2d0a4ceef025fd5a607956d53d/family1 (new location: hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family1/ea8349daeb6649fea9912bed6e5461c7_SeqId_29_) 2023-03-05 19:55:39,753 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testCompactionWithHFileLink__5vpdqq4g5ifvg6b86havldm4nss3tu8cvjsoltrio3jnc772o3b91n1bahv3v2ib/family1/4af89abac50a46daba2c4175180d74de 2023-03-05 19:55:39,755 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230305 table='testCompactionWithHFileLink' column='family1' 2023-03-05 19:55:39,755 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230305 table='testCompactionWithHFileLink' column='family1' 2023-03-05 19:55:39,755 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(360): Archiving compacted files. 2023-03-05 19:55:39,755 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(360): Archiving compacted files. 2023-03-05 19:55:39,757 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023030524196f3846324173b9e8c56ba282d604 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023030524196f3846324173b9e8c56ba282d604 2023-03-05 19:55:39,757 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023030501cbec120485489fa59d0ed556ae93f6 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023030501cbec120485489fa59d0ed556ae93f6 2023-03-05 19:55:39,758 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202303056d7220390ee448d6a463d2fc4c42f4da to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202303056d7220390ee448d6a463d2fc4c42f4da 2023-03-05 19:55:39,758 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230305202e97351c394119851f4075ef789695 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230305202e97351c394119851f4075ef789695 2023-03-05 19:55:39,759 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202303057212e8b9c498437fb3c20726fe55a714 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202303057212e8b9c498437fb3c20726fe55a714 2023-03-05 19:55:39,759 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202303058002b27e00b04b36af1ade5009390107 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202303058002b27e00b04b36af1ade5009390107 2023-03-05 19:55:39,760 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202303058825f16fb4854d6eb896c400e4d85db8 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202303058825f16fb4854d6eb896c400e4d85db8 2023-03-05 19:55:39,760 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230305fae8652a174c472c8be7df36b3f66be3 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230305fae8652a174c472c8be7df36b3f66be3 2023-03-05 19:55:39,760 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-03-05 19:55:39,760 DEBUG [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230305' 2023-03-05 19:55:39,760 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-03-05 19:55:39,760 DEBUG [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230305' 2023-03-05 19:55:39,760 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230305 for table='testCompactionWithHFileLink' and column='family1' 2023-03-05 19:55:39,760 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c4ca4238a0b923820dcc509a6f75849b20230305 for table='testCompactionWithHFileLink' and column='family1' 2023-03-05 19:55:39,760 INFO [Listener at localhost/40787] compactions.PartitionedMobCompactor(354): After compaction, there are 3 mob files. table='testCompactionWithHFileLink' column='family1' 2023-03-05 19:55:39,760 INFO [Listener at localhost/40787] compactions.PartitionedMobCompactor(364): After a mob compaction with all files selected, archiving the del files for table='testCompactionWithHFileLink' and column='family1' 2023-03-05 19:55:39,761 INFO [Listener at localhost/40787] compactions.PartitionedMobCompactor(367): [hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230305742b208c36f540e5b62794234602a315_del] 2023-03-05 19:55:39,761 DEBUG [Listener at localhost/40787] backup.HFileArchiver(360): Archiving compacted files. 2023-03-05 19:55:39,762 DEBUG [Listener at localhost/40787] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230305742b208c36f540e5b62794234602a315_del to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230305742b208c36f540e5b62794234602a315_del 2023-03-05 19:55:39,762 INFO [Listener at localhost/40787] mob.MobUtils(508): Table testCompactionWithHFileLink 1 expired mob files are deleted 2023-03-05 19:55:39,762 INFO [Listener at localhost/40787] compactions.PartitionedMobCompactor(367): [hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230305859844562c574340a9dc77e79ba2e662_del] 2023-03-05 19:55:39,762 DEBUG [Listener at localhost/40787] backup.HFileArchiver(360): Archiving compacted files. 2023-03-05 19:55:39,764 DEBUG [Listener at localhost/40787] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230305859844562c574340a9dc77e79ba2e662_del to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230305859844562c574340a9dc77e79ba2e662_del 2023-03-05 19:55:39,764 INFO [Listener at localhost/40787] mob.MobUtils(508): Table testCompactionWithHFileLink 1 expired mob files are deleted 2023-03-05 19:55:39,764 INFO [Listener at localhost/40787] compactions.PartitionedMobCompactor(367): [hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023030594dac2d83eea492f86f773d941f88e34_del] 2023-03-05 19:55:39,764 DEBUG [Listener at localhost/40787] backup.HFileArchiver(360): Archiving compacted files. 2023-03-05 19:55:39,765 DEBUG [Listener at localhost/40787] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023030594dac2d83eea492f86f773d941f88e34_del to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023030594dac2d83eea492f86f773d941f88e34_del 2023-03-05 19:55:39,765 INFO [Listener at localhost/40787] mob.MobUtils(508): Table testCompactionWithHFileLink 1 expired mob files are deleted 2023-03-05 19:55:39,768 DEBUG [Listener at localhost/40787] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-05 19:55:39,770 DEBUG [Listener at localhost/40787] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-05 19:55:39,773 DEBUG [Listener at localhost/40787] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-05 19:55:39,774 DEBUG [Listener at localhost/40787] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-05 19:55:39,780 INFO [Listener at localhost/40787] client.HBaseAdmin$15(890): Started disable of testCompactionWithHFileLink 2023-03-05 19:55:39,781 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testCompactionWithHFileLink 2023-03-05 19:55:39,782 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure2.ProcedureExecutor(1029): Stored pid=88, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testCompactionWithHFileLink 2023-03-05 19:55:39,784 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-03-05 19:55:39,784 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678046139784"}]},"ts":"1678046139784"} 2023-03-05 19:55:39,785 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=DISABLING in hbase:meta 2023-03-05 19:55:39,787 INFO [PEWorker-4] procedure.DisableTableProcedure(293): Set testCompactionWithHFileLink to state=DISABLING 2023-03-05 19:55:39,788 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=89, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=daa57ebba7c140c319a2f6facc725e4a, UNASSIGN}, {pid=90, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=942afd2d0a4ceef025fd5a607956d53d, UNASSIGN}, {pid=91, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=25af79a83c9b248c956b57fc8507ead6, UNASSIGN}] 2023-03-05 19:55:39,789 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=91, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=25af79a83c9b248c956b57fc8507ead6, UNASSIGN 2023-03-05 19:55:39,789 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=89, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=daa57ebba7c140c319a2f6facc725e4a, UNASSIGN 2023-03-05 19:55:39,790 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=90, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=942afd2d0a4ceef025fd5a607956d53d, UNASSIGN 2023-03-05 19:55:39,790 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=89 updating hbase:meta row=daa57ebba7c140c319a2f6facc725e4a, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:39,790 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=91 updating hbase:meta row=25af79a83c9b248c956b57fc8507ead6, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:39,790 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,,1678046116082.daa57ebba7c140c319a2f6facc725e4a.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678046139790"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678046139790"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678046139790"}]},"ts":"1678046139790"} 2023-03-05 19:55:39,790 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=942afd2d0a4ceef025fd5a607956d53d, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:39,790 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,2,1678046116082.25af79a83c9b248c956b57fc8507ead6.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678046139790"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678046139790"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678046139790"}]},"ts":"1678046139790"} 2023-03-05 19:55:39,790 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,1,1678046116082.942afd2d0a4ceef025fd5a607956d53d.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1678046139790"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678046139790"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678046139790"}]},"ts":"1678046139790"} 2023-03-05 19:55:39,791 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=92, ppid=89, state=RUNNABLE; CloseRegionProcedure daa57ebba7c140c319a2f6facc725e4a, server=jenkins-hbase4.apache.org,37037,1678046018983}] 2023-03-05 19:55:39,792 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=93, ppid=91, state=RUNNABLE; CloseRegionProcedure 25af79a83c9b248c956b57fc8507ead6, server=jenkins-hbase4.apache.org,37037,1678046018983}] 2023-03-05 19:55:39,793 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=94, ppid=90, state=RUNNABLE; CloseRegionProcedure 942afd2d0a4ceef025fd5a607956d53d, server=jenkins-hbase4.apache.org,37037,1678046018983}] 2023-03-05 19:55:39,885 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-03-05 19:55:39,943 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 25af79a83c9b248c956b57fc8507ead6 2023-03-05 19:55:39,943 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 25af79a83c9b248c956b57fc8507ead6, disabling compactions & flushes 2023-03-05 19:55:39,943 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,2,1678046116082.25af79a83c9b248c956b57fc8507ead6. 2023-03-05 19:55:39,943 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,2,1678046116082.25af79a83c9b248c956b57fc8507ead6. 2023-03-05 19:55:39,943 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,2,1678046116082.25af79a83c9b248c956b57fc8507ead6. after waiting 0 ms 2023-03-05 19:55:39,943 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,2,1678046116082.25af79a83c9b248c956b57fc8507ead6. 2023-03-05 19:55:39,947 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/recovered.edits/31.seqid, newMaxSeqId=31, maxSeqId=1 2023-03-05 19:55:39,948 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,2,1678046116082.25af79a83c9b248c956b57fc8507ead6. 2023-03-05 19:55:39,948 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 25af79a83c9b248c956b57fc8507ead6: 2023-03-05 19:55:39,950 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 25af79a83c9b248c956b57fc8507ead6 2023-03-05 19:55:39,950 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close daa57ebba7c140c319a2f6facc725e4a 2023-03-05 19:55:39,950 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing daa57ebba7c140c319a2f6facc725e4a, disabling compactions & flushes 2023-03-05 19:55:39,950 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,,1678046116082.daa57ebba7c140c319a2f6facc725e4a. 2023-03-05 19:55:39,950 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,,1678046116082.daa57ebba7c140c319a2f6facc725e4a. 2023-03-05 19:55:39,950 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,,1678046116082.daa57ebba7c140c319a2f6facc725e4a. after waiting 0 ms 2023-03-05 19:55:39,950 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,,1678046116082.daa57ebba7c140c319a2f6facc725e4a. 2023-03-05 19:55:39,950 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=91 updating hbase:meta row=25af79a83c9b248c956b57fc8507ead6, regionState=CLOSED 2023-03-05 19:55:39,950 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,2,1678046116082.25af79a83c9b248c956b57fc8507ead6.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678046139950"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678046139950"}]},"ts":"1678046139950"} 2023-03-05 19:55:39,953 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=93, resume processing ppid=91 2023-03-05 19:55:39,953 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=93, ppid=91, state=SUCCESS; CloseRegionProcedure 25af79a83c9b248c956b57fc8507ead6, server=jenkins-hbase4.apache.org,37037,1678046018983 in 160 msec 2023-03-05 19:55:39,954 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/recovered.edits/31.seqid, newMaxSeqId=31, maxSeqId=1 2023-03-05 19:55:39,955 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,,1678046116082.daa57ebba7c140c319a2f6facc725e4a. 2023-03-05 19:55:39,955 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for daa57ebba7c140c319a2f6facc725e4a: 2023-03-05 19:55:39,955 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=91, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=25af79a83c9b248c956b57fc8507ead6, UNASSIGN in 165 msec 2023-03-05 19:55:39,956 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed daa57ebba7c140c319a2f6facc725e4a 2023-03-05 19:55:39,956 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 942afd2d0a4ceef025fd5a607956d53d 2023-03-05 19:55:39,956 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 942afd2d0a4ceef025fd5a607956d53d, disabling compactions & flushes 2023-03-05 19:55:39,956 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,1,1678046116082.942afd2d0a4ceef025fd5a607956d53d. 2023-03-05 19:55:39,956 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,1,1678046116082.942afd2d0a4ceef025fd5a607956d53d. 2023-03-05 19:55:39,956 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,1,1678046116082.942afd2d0a4ceef025fd5a607956d53d. after waiting 0 ms 2023-03-05 19:55:39,956 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,1,1678046116082.942afd2d0a4ceef025fd5a607956d53d. 2023-03-05 19:55:39,956 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=89 updating hbase:meta row=daa57ebba7c140c319a2f6facc725e4a, regionState=CLOSED 2023-03-05 19:55:39,956 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,,1678046116082.daa57ebba7c140c319a2f6facc725e4a.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678046139956"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678046139956"}]},"ts":"1678046139956"} 2023-03-05 19:55:39,959 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=92, resume processing ppid=89 2023-03-05 19:55:39,959 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=92, ppid=89, state=SUCCESS; CloseRegionProcedure daa57ebba7c140c319a2f6facc725e4a, server=jenkins-hbase4.apache.org,37037,1678046018983 in 167 msec 2023-03-05 19:55:39,960 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/recovered.edits/31.seqid, newMaxSeqId=31, maxSeqId=1 2023-03-05 19:55:39,960 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=89, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=daa57ebba7c140c319a2f6facc725e4a, UNASSIGN in 171 msec 2023-03-05 19:55:39,960 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,1,1678046116082.942afd2d0a4ceef025fd5a607956d53d. 2023-03-05 19:55:39,960 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 942afd2d0a4ceef025fd5a607956d53d: 2023-03-05 19:55:39,961 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 942afd2d0a4ceef025fd5a607956d53d 2023-03-05 19:55:39,962 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=942afd2d0a4ceef025fd5a607956d53d, regionState=CLOSED 2023-03-05 19:55:39,962 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,1,1678046116082.942afd2d0a4ceef025fd5a607956d53d.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1678046139962"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678046139962"}]},"ts":"1678046139962"} 2023-03-05 19:55:39,963 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=94, resume processing ppid=90 2023-03-05 19:55:39,964 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=94, ppid=90, state=SUCCESS; CloseRegionProcedure 942afd2d0a4ceef025fd5a607956d53d, server=jenkins-hbase4.apache.org,37037,1678046018983 in 171 msec 2023-03-05 19:55:39,964 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=90, resume processing ppid=88 2023-03-05 19:55:39,965 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=90, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=942afd2d0a4ceef025fd5a607956d53d, UNASSIGN in 176 msec 2023-03-05 19:55:39,965 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678046139965"}]},"ts":"1678046139965"} 2023-03-05 19:55:39,966 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=DISABLED in hbase:meta 2023-03-05 19:55:39,967 INFO [PEWorker-5] procedure.DisableTableProcedure(305): Set testCompactionWithHFileLink to state=DISABLED 2023-03-05 19:55:39,969 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=88, state=SUCCESS; DisableTableProcedure table=testCompactionWithHFileLink in 187 msec 2023-03-05 19:55:40,086 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-03-05 19:55:40,086 INFO [Listener at localhost/40787] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testCompactionWithHFileLink, procId: 88 completed 2023-03-05 19:55:40,096 INFO [Listener at localhost/40787] client.HBaseAdmin(2586): Taking restore-failsafe snapshot: hbase-failsafe-snaptb-1678046125369-1678046140096 2023-03-05 19:55:40,096 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1580): Client=jenkins//172.31.14.131 snapshot request for:{ ss=hbase-failsafe-snaptb-1678046125369-1678046140096 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-03-05 19:55:40,096 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] snapshot.SnapshotDescriptionUtils(316): Creation time not specified, setting to:1678046140096 (current time:1678046140096). 2023-03-05 19:55:40,096 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] snapshot.SnapshotDescriptionUtils(332): Snapshot current TTL value: 0 resetting it to default value: 0 2023-03-05 19:55:40,097 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] zookeeper.ReadOnlyZKClient(139): Connect 0x5b3f3ac3 to 127.0.0.1:62957 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-05 19:55:40,102 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@539794c1, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-05 19:55:40,103 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-05 19:55:40,105 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:38256, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-05 19:55:40,105 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x5b3f3ac3 to 127.0.0.1:62957 2023-03-05 19:55:40,105 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-05 19:55:40,106 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] snapshot.SnapshotManager(601): No existing snapshot, attempting snapshot... 2023-03-05 19:55:40,106 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] snapshot.SnapshotManager(659): Table is disabled, running snapshot entirely on master for { ss=hbase-failsafe-snaptb-1678046125369-1678046140096 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-03-05 19:55:40,108 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure2.ProcedureExecutor(1029): Stored pid=95, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-03-05 19:55:40,108 DEBUG [PEWorker-1] locking.LockProcedure(309): LOCKED pid=95, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-03-05 19:55:40,109 INFO [PEWorker-1] procedure2.TimeoutExecutorThread(81): ADDED pid=95, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE; timeout=600000, timestamp=1678046740109 2023-03-05 19:55:40,109 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] snapshot.SnapshotManager(664): Started snapshot: { ss=hbase-failsafe-snaptb-1678046125369-1678046140096 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-03-05 19:55:40,109 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(174): Running DISABLED table snapshot hbase-failsafe-snaptb-1678046125369-1678046140096 C_M_SNAPSHOT_TABLE on table testCompactionWithHFileLink 2023-03-05 19:55:40,109 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2418): Waiting a max of 300000 ms for snapshot '{ ss=hbase-failsafe-snaptb-1678046125369-1678046140096 table=testCompactionWithHFileLink type=FLUSH ttl=0 }'' to complete. (max 300000 ms per retry) 2023-03-05 19:55:40,109 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2428): (#1) Sleeping: 100ms while waiting for snapshot completion. 2023-03-05 19:55:40,210 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-05 19:55:40,211 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1678046125369-1678046140096 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-05 19:55:40,211 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] snapshot.SnapshotManager(404): Snapshoting '{ ss=hbase-failsafe-snaptb-1678046125369-1678046140096 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-03-05 19:55:40,211 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2428): (#2) Sleeping: 200ms while waiting for snapshot completion. 2023-03-05 19:55:40,411 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-05 19:55:40,412 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1678046125369-1678046140096 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-05 19:55:40,412 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] snapshot.SnapshotManager(404): Snapshoting '{ ss=hbase-failsafe-snaptb-1678046125369-1678046140096 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-03-05 19:55:40,412 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2428): (#3) Sleeping: 300ms while waiting for snapshot completion. 2023-03-05 19:55:40,515 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.DisabledTableSnapshotHandler(97): Starting to write region info and WALs for regions for offline snapshot:{ ss=hbase-failsafe-snaptb-1678046125369-1678046140096 table=testCompactionWithHFileLink type=DISABLED ttl=0 } 2023-03-05 19:55:40,517 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(297): Storing region-info for snapshot. 2023-03-05 19:55:40,517 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(297): Storing region-info for snapshot. 2023-03-05 19:55:40,517 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(297): Storing region-info for snapshot. 2023-03-05 19:55:40,517 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(297): Storing region-info for snapshot. 2023-03-05 19:55:40,517 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(302): Creating references for hfiles 2023-03-05 19:55:40,517 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(302): Creating references for hfiles 2023-03-05 19:55:40,517 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(302): Creating references for hfiles 2023-03-05 19:55:40,517 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(302): Creating references for hfiles 2023-03-05 19:55:40,520 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family1/0a08f6f51c1546e184b4393a89afb484, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family1/b8fa05b9e21c43f2b863d96067141cb0_SeqId_29_] hfiles 2023-03-05 19:55:40,520 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (1/2): hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family1/0a08f6f51c1546e184b4393a89afb484 2023-03-05 19:55:40,520 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family1/ea8349daeb6649fea9912bed6e5461c7_SeqId_29_, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family1/ec9c98101b6045a7ae5e03e71336653c] hfiles 2023-03-05 19:55:40,520 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (2/2): hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family1/b8fa05b9e21c43f2b863d96067141cb0_SeqId_29_ 2023-03-05 19:55:40,520 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(363): Adding reference for hfile (1/2): hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family1/ea8349daeb6649fea9912bed6e5461c7_SeqId_29_ 2023-03-05 19:55:40,520 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family1/301c6a15916c4e289e14b873cb6f18fc_SeqId_29_, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family1/531e515ba5ab40f0913a9ad406709567] hfiles 2023-03-05 19:55:40,521 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(363): Adding reference for hfile (2/2): hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family1/ec9c98101b6045a7ae5e03e71336653c 2023-03-05 19:55:40,520 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230305d8d40ebe0cf6478b93eb284f958283c5, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230305ea913a4b69504c818e9b254877d6ee2b, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202303053423757e17c645b0813f1ec26bbf3247] hfiles 2023-03-05 19:55:40,521 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(363): Adding reference for hfile (1/2): hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family1/301c6a15916c4e289e14b873cb6f18fc_SeqId_29_ 2023-03-05 19:55:40,521 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (1/3): hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230305d8d40ebe0cf6478b93eb284f958283c5 2023-03-05 19:55:40,521 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(363): Adding reference for hfile (2/2): hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family1/531e515ba5ab40f0913a9ad406709567 2023-03-05 19:55:40,521 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (2/3): hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230305ea913a4b69504c818e9b254877d6ee2b 2023-03-05 19:55:40,521 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (3/3): hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202303053423757e17c645b0813f1ec26bbf3247 2023-03-05 19:55:40,521 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family2/8233b8c3ed9a4588916d13d67112fe6e] hfiles 2023-03-05 19:55:40,521 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (1/1): hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family2/8233b8c3ed9a4588916d13d67112fe6e 2023-03-05 19:55:40,522 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family2/e2d807631cc1463ab28bd4f9dd62e867] hfiles 2023-03-05 19:55:40,522 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(363): Adding reference for hfile (1/1): hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family2/e2d807631cc1463ab28bd4f9dd62e867 2023-03-05 19:55:40,522 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family2/bd91cfa3cac24d97bd240ffd35d114aa] hfiles 2023-03-05 19:55:40,522 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(363): Adding reference for hfile (1/1): hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family2/bd91cfa3cac24d97bd240ffd35d114aa 2023-03-05 19:55:40,526 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202303050699e767fa524db5b06d769d9b9ebc57, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202303052c52d973fc3d4c1d97d47e99d0c89ee5, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023030550ddbcf7ec204ae2a27dd1556aa58a72, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202303056312607d0a274d938258ee9920246856, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230305a1ce4fce5ac245639bc446818ed6cf66_del, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230305a54669736605415e9bffb71bcfa30d8e, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230305b01248d686924c22a35fa5280ae31538_del, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230305b278cd9c1bc2498a99ea539cbb9fc6a0, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230305e700388d6bf8428c86eb8d457964da6e, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230305f7d80ee747a04a6ca514f2b4ec56d1a4, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202303050e8f89505ca74920aa760c9a68a9cea6_del, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023030553b3fcde254241d4b99a405451edd672, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202303058a1c1fcb520745f8bb661ce5d25985d3, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230305b7c7c8f3cd6343f9870b0f76ea40a2c6, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230305dba00f651aab47a28da5218ebaba00df] hfiles 2023-03-05 19:55:40,526 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (1/15): hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202303050699e767fa524db5b06d769d9b9ebc57 2023-03-05 19:55:40,526 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (2/15): hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202303052c52d973fc3d4c1d97d47e99d0c89ee5 2023-03-05 19:55:40,526 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (3/15): hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023030550ddbcf7ec204ae2a27dd1556aa58a72 2023-03-05 19:55:40,526 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (4/15): hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202303056312607d0a274d938258ee9920246856 2023-03-05 19:55:40,526 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (5/15): hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230305a1ce4fce5ac245639bc446818ed6cf66_del 2023-03-05 19:55:40,526 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (6/15): hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230305a54669736605415e9bffb71bcfa30d8e 2023-03-05 19:55:40,526 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (7/15): hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230305b01248d686924c22a35fa5280ae31538_del 2023-03-05 19:55:40,526 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (8/15): hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230305b278cd9c1bc2498a99ea539cbb9fc6a0 2023-03-05 19:55:40,526 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (9/15): hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230305e700388d6bf8428c86eb8d457964da6e 2023-03-05 19:55:40,526 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (10/15): hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230305f7d80ee747a04a6ca514f2b4ec56d1a4 2023-03-05 19:55:40,526 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (11/15): hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202303050e8f89505ca74920aa760c9a68a9cea6_del 2023-03-05 19:55:40,526 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (12/15): hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023030553b3fcde254241d4b99a405451edd672 2023-03-05 19:55:40,526 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (13/15): hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202303058a1c1fcb520745f8bb661ce5d25985d3 2023-03-05 19:55:40,526 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (14/15): hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230305b7c7c8f3cd6343f9870b0f76ea40a2c6 2023-03-05 19:55:40,526 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (15/15): hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230305dba00f651aab47a28da5218ebaba00df 2023-03-05 19:55:40,712 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-05 19:55:40,713 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1678046125369-1678046140096 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-05 19:55:40,713 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] snapshot.SnapshotManager(404): Snapshoting '{ ss=hbase-failsafe-snaptb-1678046125369-1678046140096 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-03-05 19:55:40,713 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2428): (#4) Sleeping: 500ms while waiting for snapshot completion. 2023-03-05 19:55:40,935 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.DisabledTableSnapshotHandler(120): Marking snapshot{ ss=hbase-failsafe-snaptb-1678046125369-1678046140096 table=testCompactionWithHFileLink type=DISABLED ttl=0 } as finished. 2023-03-05 19:55:40,935 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(484): Convert to Single Snapshot Manifest for hbase-failsafe-snaptb-1678046125369-1678046140096 2023-03-05 19:55:40,936 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifestV1(126): No regions under directory:hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1678046125369-1678046140096 2023-03-05 19:55:41,200 DEBUG [regionserver/jenkins-hbase4:0.Chore.2] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-03-05 19:55:41,213 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-05 19:55:41,214 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1678046125369-1678046140096 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-05 19:55:41,214 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] snapshot.SnapshotManager(404): Snapshoting '{ ss=hbase-failsafe-snaptb-1678046125369-1678046140096 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-03-05 19:55:41,214 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2428): (#5) Sleeping: 1000ms while waiting for snapshot completion. 2023-03-05 19:55:41,357 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotDescriptionUtils(404): Sentinel is done, just moving the snapshot from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1678046125369-1678046140096 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/.hbase-snapshot/hbase-failsafe-snaptb-1678046125369-1678046140096 2023-03-05 19:55:41,373 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(229): Snapshot hbase-failsafe-snaptb-1678046125369-1678046140096 of table testCompactionWithHFileLink completed 2023-03-05 19:55:41,373 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(246): Launching cleanup of working dir:hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1678046125369-1678046140096 2023-03-05 19:55:41,373 ERROR [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(251): Couldn't delete snapshot working directory:hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1678046125369-1678046140096 2023-03-05 19:55:41,373 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(257): Table snapshot journal : Running DISABLED table snapshot hbase-failsafe-snaptb-1678046125369-1678046140096 C_M_SNAPSHOT_TABLE on table testCompactionWithHFileLink at 1678046140109Starting to write region info and WALs for regions for offline snapshot:{ ss=hbase-failsafe-snaptb-1678046125369-1678046140096 table=testCompactionWithHFileLink type=DISABLED ttl=0 } at 1678046140515 (+406 ms)Consolidate snapshot: hbase-failsafe-snaptb-1678046125369-1678046140096 at 1678046140935 (+420 ms)Loading Region manifests for hbase-failsafe-snaptb-1678046125369-1678046140096 at 1678046140935Writing data manifest for hbase-failsafe-snaptb-1678046125369-1678046140096 at 1678046140938 (+3 ms)Verifying snapshot: hbase-failsafe-snaptb-1678046125369-1678046140096 at 1678046141344 (+406 ms)Snapshot hbase-failsafe-snaptb-1678046125369-1678046140096 of table testCompactionWithHFileLink completed at 1678046141373 (+29 ms) 2023-03-05 19:55:41,374 DEBUG [PEWorker-4] locking.LockProcedure(242): UNLOCKED pid=95, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-03-05 19:55:41,376 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=95, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE in 1.2680 sec 2023-03-05 19:55:42,214 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-05 19:55:42,215 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1678046125369-1678046140096 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-05 19:55:42,215 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] snapshot.SnapshotManager(401): Snapshot '{ ss=hbase-failsafe-snaptb-1678046125369-1678046140096 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' has completed, notifying client. 2023-03-05 19:55:42,252 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure2.ProcedureExecutor(1029): Stored pid=96, state=RUNNABLE:RESTORE_SNAPSHOT_PRE_OPERATION; RestoreSnapshotProcedure (table=testCompactionWithHFileLink snapshot=name: "snaptb-1678046125369" table: "testCompactionWithHFileLink" creation_time: 1678046125378 type: FLUSH version: 2 ttl: 0 ) 2023-03-05 19:55:42,252 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] snapshot.SnapshotManager(880): Restore snapshot=snaptb-1678046125369 as table=testCompactionWithHFileLink 2023-03-05 19:55:42,255 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-03-05 19:55:42,356 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-03-05 19:55:42,557 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-03-05 19:55:42,661 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/.tabledesc/.tableinfo.0000000002 2023-03-05 19:55:42,662 DEBUG [PEWorker-2] util.FSTableDescriptors(519): Deleted hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/.tabledesc/.tableinfo.0000000001 2023-03-05 19:55:42,662 INFO [PEWorker-2] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/.tabledesc/.tableinfo.0000000002 2023-03-05 19:55:42,663 INFO [PEWorker-2] procedure.RestoreSnapshotProcedure(374): Starting restore snapshot={ ss=snaptb-1678046125369 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-03-05 19:55:42,667 INFO [PEWorker-2] snapshot.RestoreSnapshotHelper(177): starting restore table regions using snapshot=name: "snaptb-1678046125369" table: "testCompactionWithHFileLink" creation_time: 1678046125378 type: FLUSH version: 2 ttl: 0 2023-03-05 19:55:42,668 DEBUG [PEWorker-2] snapshot.RestoreSnapshotHelper(785): get table regions: hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink 2023-03-05 19:55:42,672 DEBUG [PEWorker-2] snapshot.RestoreSnapshotHelper(797): found 3 regions for table=testCompactionWithHFileLink 2023-03-05 19:55:42,675 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230305d8d40ebe0cf6478b93eb284f958283c5 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230305d8d40ebe0cf6478b93eb284f958283c5 2023-03-05 19:55:42,676 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230305ea913a4b69504c818e9b254877d6ee2b to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230305ea913a4b69504c818e9b254877d6ee2b 2023-03-05 19:55:42,677 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202303053423757e17c645b0813f1ec26bbf3247 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202303053423757e17c645b0813f1ec26bbf3247 2023-03-05 19:55:42,677 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c4ca4238a0b923820dcc509a6f75849b2023030501cbec120485489fa59d0ed556ae93f6 of snapshot=snaptb-1678046125369 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-05 19:55:42,680 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c4ca4238a0b923820dcc509a6f75849b20230305202e97351c394119851f4075ef789695 of snapshot=snaptb-1678046125369 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-05 19:55:42,683 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c4ca4238a0b923820dcc509a6f75849b202303058002b27e00b04b36af1ade5009390107 of snapshot=snaptb-1678046125369 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-05 19:55:42,686 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c4ca4238a0b923820dcc509a6f75849b20230305fae8652a174c472c8be7df36b3f66be3 of snapshot=snaptb-1678046125369 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-05 19:55:42,688 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c81e728d9d4c2f636f067f89cc14862c2023030524196f3846324173b9e8c56ba282d604 of snapshot=snaptb-1678046125369 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-05 19:55:42,691 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c81e728d9d4c2f636f067f89cc14862c202303056d7220390ee448d6a463d2fc4c42f4da of snapshot=snaptb-1678046125369 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-05 19:55:42,694 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c81e728d9d4c2f636f067f89cc14862c202303057212e8b9c498437fb3c20726fe55a714 of snapshot=snaptb-1678046125369 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-05 19:55:42,696 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c81e728d9d4c2f636f067f89cc14862c202303058825f16fb4854d6eb896c400e4d85db8 of snapshot=snaptb-1678046125369 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-05 19:55:42,699 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d41d8cd98f00b204e9800998ecf8427e202303056a09d4fe8b054b179c0313b89fbc21ce of snapshot=snaptb-1678046125369 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-05 19:55:42,701 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d41d8cd98f00b204e9800998ecf8427e20230305a33235cee5224842811abef82371aea6 of snapshot=snaptb-1678046125369 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-05 19:55:42,704 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d41d8cd98f00b204e9800998ecf8427e20230305da73569395114dad837fdbde0dcbba50 of snapshot=snaptb-1678046125369 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-05 19:55:42,707 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d41d8cd98f00b204e9800998ecf8427e20230305f43d564866564d9e939ce4a0e9ba6b2f of snapshot=snaptb-1678046125369 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-05 19:55:42,711 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230305a1ce4fce5ac245639bc446818ed6cf66_del to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230305a1ce4fce5ac245639bc446818ed6cf66_del 2023-03-05 19:55:42,712 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202303050e8f89505ca74920aa760c9a68a9cea6_del to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202303050e8f89505ca74920aa760c9a68a9cea6_del 2023-03-05 19:55:42,713 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230305b01248d686924c22a35fa5280ae31538_del to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230305b01248d686924c22a35fa5280ae31538_del 2023-03-05 19:55:42,713 INFO [PEWorker-2] snapshot.RestoreSnapshotHelper(223): region to restore: 25af79a83c9b248c956b57fc8507ead6 2023-03-05 19:55:42,713 INFO [PEWorker-2] snapshot.RestoreSnapshotHelper(223): region to restore: 942afd2d0a4ceef025fd5a607956d53d 2023-03-05 19:55:42,713 INFO [PEWorker-2] snapshot.RestoreSnapshotHelper(223): region to restore: daa57ebba7c140c319a2f6facc725e4a 2023-03-05 19:55:42,716 DEBUG [RestoreSnapshot-pool-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family1/b8fa05b9e21c43f2b863d96067141cb0_SeqId_29_ to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family1/b8fa05b9e21c43f2b863d96067141cb0_SeqId_29_ 2023-03-05 19:55:42,716 DEBUG [RestoreSnapshot-pool-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family1/301c6a15916c4e289e14b873cb6f18fc_SeqId_29_ to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family1/301c6a15916c4e289e14b873cb6f18fc_SeqId_29_ 2023-03-05 19:55:42,716 DEBUG [RestoreSnapshot-pool-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family1/ea8349daeb6649fea9912bed6e5461c7_SeqId_29_ to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family1/ea8349daeb6649fea9912bed6e5461c7_SeqId_29_ 2023-03-05 19:55:42,717 DEBUG [RestoreSnapshot-pool-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family1/0a08f6f51c1546e184b4393a89afb484 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family1/0a08f6f51c1546e184b4393a89afb484 2023-03-05 19:55:42,717 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 9a9155a47d074f5f9005ddf6337e173e of snapshot=snaptb-1678046125369 to region=25af79a83c9b248c956b57fc8507ead6 table=testCompactionWithHFileLink 2023-03-05 19:55:42,717 DEBUG [RestoreSnapshot-pool-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family1/ec9c98101b6045a7ae5e03e71336653c to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family1/ec9c98101b6045a7ae5e03e71336653c 2023-03-05 19:55:42,717 DEBUG [RestoreSnapshot-pool-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family1/531e515ba5ab40f0913a9ad406709567 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family1/531e515ba5ab40f0913a9ad406709567 2023-03-05 19:55:42,718 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink cdcb9dd831794696af2aaf30754840ee of snapshot=snaptb-1678046125369 to region=daa57ebba7c140c319a2f6facc725e4a table=testCompactionWithHFileLink 2023-03-05 19:55:42,717 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 2ddb935fe9dc46778a4c450ff10d3c44 of snapshot=snaptb-1678046125369 to region=942afd2d0a4ceef025fd5a607956d53d table=testCompactionWithHFileLink 2023-03-05 19:55:42,726 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 24f57a619907417b8e8a0222c800bc07 of snapshot=snaptb-1678046125369 to region=25af79a83c9b248c956b57fc8507ead6 table=testCompactionWithHFileLink 2023-03-05 19:55:42,726 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink f6b69932b5da4e0d9155ef905eacf40c of snapshot=snaptb-1678046125369 to region=daa57ebba7c140c319a2f6facc725e4a table=testCompactionWithHFileLink 2023-03-05 19:55:42,726 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink dc8108fdcc6a4a0a9720b3e2336cb620 of snapshot=snaptb-1678046125369 to region=942afd2d0a4ceef025fd5a607956d53d table=testCompactionWithHFileLink 2023-03-05 19:55:42,729 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 43db0ea3bd404e2fa10949ad9df289b4 of snapshot=snaptb-1678046125369 to region=daa57ebba7c140c319a2f6facc725e4a table=testCompactionWithHFileLink 2023-03-05 19:55:42,729 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 3004199e82364ee9abaf18f0f843fc76 of snapshot=snaptb-1678046125369 to region=25af79a83c9b248c956b57fc8507ead6 table=testCompactionWithHFileLink 2023-03-05 19:55:42,729 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c88f1bd396f4409ca38794d504f5c156 of snapshot=snaptb-1678046125369 to region=942afd2d0a4ceef025fd5a607956d53d table=testCompactionWithHFileLink 2023-03-05 19:55:42,732 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink e02bc7cebc7e44af831019cee45cd29b of snapshot=snaptb-1678046125369 to region=daa57ebba7c140c319a2f6facc725e4a table=testCompactionWithHFileLink 2023-03-05 19:55:42,732 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 6b8608e70a864d6e8a4e55c85261adfe of snapshot=snaptb-1678046125369 to region=25af79a83c9b248c956b57fc8507ead6 table=testCompactionWithHFileLink 2023-03-05 19:55:42,733 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 8139f54670e14aa3a572f9bdb2477108 of snapshot=snaptb-1678046125369 to region=942afd2d0a4ceef025fd5a607956d53d table=testCompactionWithHFileLink 2023-03-05 19:55:42,742 DEBUG [RestoreSnapshot-pool-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family2/bd91cfa3cac24d97bd240ffd35d114aa to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family2/bd91cfa3cac24d97bd240ffd35d114aa 2023-03-05 19:55:42,742 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c42309b45f3243f2a021144945ad30f4 of snapshot=snaptb-1678046125369 to region=daa57ebba7c140c319a2f6facc725e4a table=testCompactionWithHFileLink 2023-03-05 19:55:42,742 DEBUG [RestoreSnapshot-pool-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family2/8233b8c3ed9a4588916d13d67112fe6e to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family2/8233b8c3ed9a4588916d13d67112fe6e 2023-03-05 19:55:42,742 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 22cbfc0fa2af442f976458cd740e398d of snapshot=snaptb-1678046125369 to region=25af79a83c9b248c956b57fc8507ead6 table=testCompactionWithHFileLink 2023-03-05 19:55:42,742 DEBUG [RestoreSnapshot-pool-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family2/e2d807631cc1463ab28bd4f9dd62e867 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family2/e2d807631cc1463ab28bd4f9dd62e867 2023-03-05 19:55:42,742 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 9a13755a01ce4e01b7e321e95b43b8cc of snapshot=snaptb-1678046125369 to region=942afd2d0a4ceef025fd5a607956d53d table=testCompactionWithHFileLink 2023-03-05 19:55:42,749 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c2d4bdd0af534bf8b270644df2f98eaf of snapshot=snaptb-1678046125369 to region=daa57ebba7c140c319a2f6facc725e4a table=testCompactionWithHFileLink 2023-03-05 19:55:42,749 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink e2b047fb6d65462d939ae96ac7375619 of snapshot=snaptb-1678046125369 to region=25af79a83c9b248c956b57fc8507ead6 table=testCompactionWithHFileLink 2023-03-05 19:55:42,750 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink af93404d81154179a4c27fce0e335552 of snapshot=snaptb-1678046125369 to region=942afd2d0a4ceef025fd5a607956d53d table=testCompactionWithHFileLink 2023-03-05 19:55:42,753 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 13ca0708b05643899307a03f0fcc261a of snapshot=snaptb-1678046125369 to region=daa57ebba7c140c319a2f6facc725e4a table=testCompactionWithHFileLink 2023-03-05 19:55:42,753 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 2c3de0e4dce5444ca51da5f66eaf06a1 of snapshot=snaptb-1678046125369 to region=942afd2d0a4ceef025fd5a607956d53d table=testCompactionWithHFileLink 2023-03-05 19:55:42,753 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 1dfce094a6164169b84ae1182f0260e1 of snapshot=snaptb-1678046125369 to region=25af79a83c9b248c956b57fc8507ead6 table=testCompactionWithHFileLink 2023-03-05 19:55:42,756 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 9173733bfa77411fa7038a412b2f97e1 of snapshot=snaptb-1678046125369 to region=daa57ebba7c140c319a2f6facc725e4a table=testCompactionWithHFileLink 2023-03-05 19:55:42,756 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 55d239cb7531440088e28c0c9e6143d3 of snapshot=snaptb-1678046125369 to region=942afd2d0a4ceef025fd5a607956d53d table=testCompactionWithHFileLink 2023-03-05 19:55:42,756 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 11fd411913d44c5187ed3f273ed56bdd of snapshot=snaptb-1678046125369 to region=25af79a83c9b248c956b57fc8507ead6 table=testCompactionWithHFileLink 2023-03-05 19:55:42,761 INFO [PEWorker-2] snapshot.RestoreSnapshotHelper(266): finishing restore table regions using snapshot=name: "snaptb-1678046125369" table: "testCompactionWithHFileLink" creation_time: 1678046125378 type: FLUSH version: 2 ttl: 0 2023-03-05 19:55:42,762 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testCompactionWithHFileLink,2,1678046116082.25af79a83c9b248c956b57fc8507ead6.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1678046142762"}]},"ts":"9223372036854775807"} 2023-03-05 19:55:42,762 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testCompactionWithHFileLink,1,1678046116082.942afd2d0a4ceef025fd5a607956d53d.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1678046142762"}]},"ts":"9223372036854775807"} 2023-03-05 19:55:42,762 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testCompactionWithHFileLink,,1678046116082.daa57ebba7c140c319a2f6facc725e4a.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1678046142762"}]},"ts":"9223372036854775807"} 2023-03-05 19:55:42,764 INFO [PEWorker-2] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-03-05 19:55:42,764 DEBUG [PEWorker-2] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 25af79a83c9b248c956b57fc8507ead6, NAME => 'testCompactionWithHFileLink,2,1678046116082.25af79a83c9b248c956b57fc8507ead6.', STARTKEY => '2', ENDKEY => ''}, {ENCODED => 942afd2d0a4ceef025fd5a607956d53d, NAME => 'testCompactionWithHFileLink,1,1678046116082.942afd2d0a4ceef025fd5a607956d53d.', STARTKEY => '1', ENDKEY => '2'}, {ENCODED => daa57ebba7c140c319a2f6facc725e4a, NAME => 'testCompactionWithHFileLink,,1678046116082.daa57ebba7c140c319a2f6facc725e4a.', STARTKEY => '', ENDKEY => '1'}] 2023-03-05 19:55:42,764 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,2,1678046116082.25af79a83c9b248c956b57fc8507ead6.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678046142763"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678046142763"}]},"ts":"1678046142763"} 2023-03-05 19:55:42,764 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,1,1678046116082.942afd2d0a4ceef025fd5a607956d53d.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1678046142763"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678046142763"}]},"ts":"1678046142763"} 2023-03-05 19:55:42,764 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,,1678046116082.daa57ebba7c140c319a2f6facc725e4a.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678046142763"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678046142763"}]},"ts":"1678046142763"} 2023-03-05 19:55:42,766 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-03-05 19:55:42,766 INFO [PEWorker-2] hbase.MetaTableAccessor(1810): Overwritten 3 regions to Meta 2023-03-05 19:55:42,766 DEBUG [PEWorker-2] hbase.MetaTableAccessor(1811): Overwritten regions: [{ENCODED => 25af79a83c9b248c956b57fc8507ead6, NAME => 'testCompactionWithHFileLink,2,1678046116082.25af79a83c9b248c956b57fc8507ead6.', STARTKEY => '2', ENDKEY => ''}, {ENCODED => 942afd2d0a4ceef025fd5a607956d53d, NAME => 'testCompactionWithHFileLink,1,1678046116082.942afd2d0a4ceef025fd5a607956d53d.', STARTKEY => '1', ENDKEY => '2'}, {ENCODED => daa57ebba7c140c319a2f6facc725e4a, NAME => 'testCompactionWithHFileLink,,1678046116082.daa57ebba7c140c319a2f6facc725e4a.', STARTKEY => '', ENDKEY => '1'}] 2023-03-05 19:55:42,767 INFO [PEWorker-2] procedure.RestoreSnapshotProcedure(447): Restore snapshot={ ss=snaptb-1678046125369 table=testCompactionWithHFileLink type=FLUSH ttl=0 } on table=testCompactionWithHFileLink completed! 2023-03-05 19:55:42,769 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=96, state=SUCCESS; RestoreSnapshotProcedure (table=testCompactionWithHFileLink snapshot=name: "snaptb-1678046125369" table: "testCompactionWithHFileLink" creation_time: 1678046125378 type: FLUSH version: 2 ttl: 0 ) in 524 msec 2023-03-05 19:55:42,858 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-03-05 19:55:42,858 INFO [Listener at localhost/40787] client.HBaseAdmin$TableFuture(3541): Operation: MODIFY, Table Name: default:testCompactionWithHFileLink, procId: 96 completed 2023-03-05 19:55:42,858 INFO [Listener at localhost/40787] client.HBaseAdmin(2618): Deleting restore-failsafe snapshot: hbase-failsafe-snaptb-1678046125369-1678046140096 2023-03-05 19:55:42,862 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(748): Client=jenkins//172.31.14.131 delete name: "hbase-failsafe-snaptb-1678046125369-1678046140096" 2023-03-05 19:55:42,864 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] snapshot.SnapshotManager(333): Deleting snapshot: hbase-failsafe-snaptb-1678046125369-1678046140096 2023-03-05 19:55:42,867 INFO [Listener at localhost/40787] client.HBaseAdmin$14(830): Started enable of testCompactionWithHFileLink 2023-03-05 19:55:42,870 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.HMaster$10(2385): Client=jenkins//172.31.14.131 enable testCompactionWithHFileLink 2023-03-05 19:55:42,876 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure2.ProcedureExecutor(1029): Stored pid=97, state=RUNNABLE:ENABLE_TABLE_PREPARE; EnableTableProcedure table=testCompactionWithHFileLink 2023-03-05 19:55:42,879 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1227): Checking to see if procedure is done pid=97 2023-03-05 19:55:42,879 INFO [PEWorker-3] procedure.EnableTableProcedure(296): Attempting to enable the table testCompactionWithHFileLink 2023-03-05 19:55:42,879 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678046142879"}]},"ts":"1678046142879"} 2023-03-05 19:55:42,880 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=ENABLING in hbase:meta 2023-03-05 19:55:42,882 DEBUG [PEWorker-3] procedure.EnableTableProcedure(107): No change in number of region replicas (configuredReplicaCount=1); assigning. 2023-03-05 19:55:42,882 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=98, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=daa57ebba7c140c319a2f6facc725e4a, ASSIGN}, {pid=99, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=942afd2d0a4ceef025fd5a607956d53d, ASSIGN}, {pid=100, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=25af79a83c9b248c956b57fc8507ead6, ASSIGN}] 2023-03-05 19:55:42,884 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=99, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=942afd2d0a4ceef025fd5a607956d53d, ASSIGN 2023-03-05 19:55:42,884 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=100, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=25af79a83c9b248c956b57fc8507ead6, ASSIGN 2023-03-05 19:55:42,884 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=98, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=daa57ebba7c140c319a2f6facc725e4a, ASSIGN 2023-03-05 19:55:42,885 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=98, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=daa57ebba7c140c319a2f6facc725e4a, ASSIGN; state=CLOSED, location=null; forceNewPlan=false, retain=false 2023-03-05 19:55:42,885 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=100, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=25af79a83c9b248c956b57fc8507ead6, ASSIGN; state=CLOSED, location=null; forceNewPlan=false, retain=false 2023-03-05 19:55:42,885 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=99, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=942afd2d0a4ceef025fd5a607956d53d, ASSIGN; state=CLOSED, location=null; forceNewPlan=false, retain=false 2023-03-05 19:55:42,980 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1227): Checking to see if procedure is done pid=97 2023-03-05 19:55:43,036 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=100 updating hbase:meta row=25af79a83c9b248c956b57fc8507ead6, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:43,036 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=99 updating hbase:meta row=942afd2d0a4ceef025fd5a607956d53d, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:43,036 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=daa57ebba7c140c319a2f6facc725e4a, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:43,036 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,2,1678046116082.25af79a83c9b248c956b57fc8507ead6.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678046143036"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678046143036"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678046143036"}]},"ts":"1678046143036"} 2023-03-05 19:55:43,037 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,,1678046116082.daa57ebba7c140c319a2f6facc725e4a.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678046143036"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678046143036"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678046143036"}]},"ts":"1678046143036"} 2023-03-05 19:55:43,036 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,1,1678046116082.942afd2d0a4ceef025fd5a607956d53d.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1678046143036"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678046143036"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678046143036"}]},"ts":"1678046143036"} 2023-03-05 19:55:43,038 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=101, ppid=100, state=RUNNABLE; OpenRegionProcedure 25af79a83c9b248c956b57fc8507ead6, server=jenkins-hbase4.apache.org,37037,1678046018983}] 2023-03-05 19:55:43,038 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=102, ppid=98, state=RUNNABLE; OpenRegionProcedure daa57ebba7c140c319a2f6facc725e4a, server=jenkins-hbase4.apache.org,37037,1678046018983}] 2023-03-05 19:55:43,039 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=103, ppid=99, state=RUNNABLE; OpenRegionProcedure 942afd2d0a4ceef025fd5a607956d53d, server=jenkins-hbase4.apache.org,37037,1678046018983}] 2023-03-05 19:55:43,181 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1227): Checking to see if procedure is done pid=97 2023-03-05 19:55:43,192 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,,1678046116082.daa57ebba7c140c319a2f6facc725e4a. 2023-03-05 19:55:43,192 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => daa57ebba7c140c319a2f6facc725e4a, NAME => 'testCompactionWithHFileLink,,1678046116082.daa57ebba7c140c319a2f6facc725e4a.', STARTKEY => '', ENDKEY => '1'} 2023-03-05 19:55:43,192 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink daa57ebba7c140c319a2f6facc725e4a 2023-03-05 19:55:43,192 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,,1678046116082.daa57ebba7c140c319a2f6facc725e4a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-05 19:55:43,193 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for daa57ebba7c140c319a2f6facc725e4a 2023-03-05 19:55:43,193 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for daa57ebba7c140c319a2f6facc725e4a 2023-03-05 19:55:43,194 INFO [StoreOpener-daa57ebba7c140c319a2f6facc725e4a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region daa57ebba7c140c319a2f6facc725e4a 2023-03-05 19:55:43,194 DEBUG [StoreOpener-daa57ebba7c140c319a2f6facc725e4a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family1 2023-03-05 19:55:43,195 DEBUG [StoreOpener-daa57ebba7c140c319a2f6facc725e4a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family1 2023-03-05 19:55:43,195 INFO [StoreOpener-daa57ebba7c140c319a2f6facc725e4a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region daa57ebba7c140c319a2f6facc725e4a columnFamilyName family1 2023-03-05 19:55:43,209 DEBUG [StoreOpener-daa57ebba7c140c319a2f6facc725e4a-1] regionserver.HStore(539): loaded hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family1/testCompactionWithHFileLink=daa57ebba7c140c319a2f6facc725e4a-43db0ea3bd404e2fa10949ad9df289b4 2023-03-05 19:55:43,213 DEBUG [StoreOpener-daa57ebba7c140c319a2f6facc725e4a-1] regionserver.HStore(539): loaded hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family1/testCompactionWithHFileLink=daa57ebba7c140c319a2f6facc725e4a-cdcb9dd831794696af2aaf30754840ee 2023-03-05 19:55:43,218 DEBUG [StoreOpener-daa57ebba7c140c319a2f6facc725e4a-1] regionserver.HStore(539): loaded hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family1/testCompactionWithHFileLink=daa57ebba7c140c319a2f6facc725e4a-e02bc7cebc7e44af831019cee45cd29b 2023-03-05 19:55:43,223 DEBUG [StoreOpener-daa57ebba7c140c319a2f6facc725e4a-1] regionserver.HStore(539): loaded hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family1/testCompactionWithHFileLink=daa57ebba7c140c319a2f6facc725e4a-f6b69932b5da4e0d9155ef905eacf40c 2023-03-05 19:55:43,223 INFO [StoreOpener-daa57ebba7c140c319a2f6facc725e4a-1] regionserver.HStore(310): Store=daa57ebba7c140c319a2f6facc725e4a/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-05 19:55:43,223 INFO [StoreOpener-daa57ebba7c140c319a2f6facc725e4a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region daa57ebba7c140c319a2f6facc725e4a 2023-03-05 19:55:43,224 DEBUG [StoreOpener-daa57ebba7c140c319a2f6facc725e4a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family2 2023-03-05 19:55:43,224 DEBUG [StoreOpener-daa57ebba7c140c319a2f6facc725e4a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family2 2023-03-05 19:55:43,224 INFO [StoreOpener-daa57ebba7c140c319a2f6facc725e4a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region daa57ebba7c140c319a2f6facc725e4a columnFamilyName family2 2023-03-05 19:55:43,231 DEBUG [StoreOpener-daa57ebba7c140c319a2f6facc725e4a-1] regionserver.HStore(539): loaded hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family2/testCompactionWithHFileLink=daa57ebba7c140c319a2f6facc725e4a-13ca0708b05643899307a03f0fcc261a 2023-03-05 19:55:43,236 DEBUG [StoreOpener-daa57ebba7c140c319a2f6facc725e4a-1] regionserver.HStore(539): loaded hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family2/testCompactionWithHFileLink=daa57ebba7c140c319a2f6facc725e4a-9173733bfa77411fa7038a412b2f97e1 2023-03-05 19:55:43,240 DEBUG [StoreOpener-daa57ebba7c140c319a2f6facc725e4a-1] regionserver.HStore(539): loaded hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family2/testCompactionWithHFileLink=daa57ebba7c140c319a2f6facc725e4a-c2d4bdd0af534bf8b270644df2f98eaf 2023-03-05 19:55:43,246 DEBUG [StoreOpener-daa57ebba7c140c319a2f6facc725e4a-1] regionserver.HStore(539): loaded hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family2/testCompactionWithHFileLink=daa57ebba7c140c319a2f6facc725e4a-c42309b45f3243f2a021144945ad30f4 2023-03-05 19:55:43,246 INFO [StoreOpener-daa57ebba7c140c319a2f6facc725e4a-1] regionserver.HStore(310): Store=daa57ebba7c140c319a2f6facc725e4a/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-05 19:55:43,247 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a 2023-03-05 19:55:43,248 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a 2023-03-05 19:55:43,250 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-05 19:55:43,251 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for daa57ebba7c140c319a2f6facc725e4a 2023-03-05 19:55:43,252 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened daa57ebba7c140c319a2f6facc725e4a; next sequenceid=32; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59449652, jitterRate=-0.1141311526298523}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-05 19:55:43,252 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for daa57ebba7c140c319a2f6facc725e4a: 2023-03-05 19:55:43,253 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,,1678046116082.daa57ebba7c140c319a2f6facc725e4a., pid=102, masterSystemTime=1678046143189 2023-03-05 19:55:43,253 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-05 19:55:43,254 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-03-05 19:55:43,254 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-03-05 19:55:43,254 DEBUG [RS:0;jenkins-hbase4:37037-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-03-05 19:55:43,255 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,,1678046116082.daa57ebba7c140c319a2f6facc725e4a. 2023-03-05 19:55:43,255 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,,1678046116082.daa57ebba7c140c319a2f6facc725e4a. 2023-03-05 19:55:43,255 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,1,1678046116082.942afd2d0a4ceef025fd5a607956d53d. 2023-03-05 19:55:43,255 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 942afd2d0a4ceef025fd5a607956d53d, NAME => 'testCompactionWithHFileLink,1,1678046116082.942afd2d0a4ceef025fd5a607956d53d.', STARTKEY => '1', ENDKEY => '2'} 2023-03-05 19:55:43,256 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=daa57ebba7c140c319a2f6facc725e4a, regionState=OPEN, openSeqNum=32, regionLocation=jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:43,256 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink 942afd2d0a4ceef025fd5a607956d53d 2023-03-05 19:55:43,256 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,,1678046116082.daa57ebba7c140c319a2f6facc725e4a.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678046143255"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678046143255"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678046143255"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678046143255"}]},"ts":"1678046143255"} 2023-03-05 19:55:43,256 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,1,1678046116082.942afd2d0a4ceef025fd5a607956d53d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-05 19:55:43,256 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 942afd2d0a4ceef025fd5a607956d53d 2023-03-05 19:55:43,256 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 942afd2d0a4ceef025fd5a607956d53d 2023-03-05 19:55:43,257 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 22152 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-03-05 19:55:43,257 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.HStore(1912): daa57ebba7c140c319a2f6facc725e4a/family1 is initiating minor compaction (all files) 2023-03-05 19:55:43,257 INFO [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of daa57ebba7c140c319a2f6facc725e4a/family1 in testCompactionWithHFileLink,,1678046116082.daa57ebba7c140c319a2f6facc725e4a. 2023-03-05 19:55:43,258 INFO [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family1/testCompactionWithHFileLink=daa57ebba7c140c319a2f6facc725e4a-cdcb9dd831794696af2aaf30754840ee, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family1/testCompactionWithHFileLink=daa57ebba7c140c319a2f6facc725e4a-f6b69932b5da4e0d9155ef905eacf40c, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family1/testCompactionWithHFileLink=daa57ebba7c140c319a2f6facc725e4a-43db0ea3bd404e2fa10949ad9df289b4, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family1/testCompactionWithHFileLink=daa57ebba7c140c319a2f6facc725e4a-e02bc7cebc7e44af831019cee45cd29b] into tmpdir=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/.tmp, totalSize=21.6 K 2023-03-05 19:55:43,258 INFO [StoreOpener-942afd2d0a4ceef025fd5a607956d53d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 942afd2d0a4ceef025fd5a607956d53d 2023-03-05 19:55:43,258 DEBUG [RS:0;jenkins-hbase4:37037-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 20920 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-03-05 19:55:43,258 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=daa57ebba7c140c319a2f6facc725e4a-cdcb9dd831794696af2aaf30754840ee, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678046118195 2023-03-05 19:55:43,258 DEBUG [RS:0;jenkins-hbase4:37037-longCompactions-0] regionserver.HStore(1912): daa57ebba7c140c319a2f6facc725e4a/family2 is initiating minor compaction (all files) 2023-03-05 19:55:43,258 INFO [RS:0;jenkins-hbase4:37037-longCompactions-0] regionserver.HRegion(2259): Starting compaction of daa57ebba7c140c319a2f6facc725e4a/family2 in testCompactionWithHFileLink,,1678046116082.daa57ebba7c140c319a2f6facc725e4a. 2023-03-05 19:55:43,259 INFO [RS:0;jenkins-hbase4:37037-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family2/testCompactionWithHFileLink=daa57ebba7c140c319a2f6facc725e4a-c42309b45f3243f2a021144945ad30f4, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family2/testCompactionWithHFileLink=daa57ebba7c140c319a2f6facc725e4a-c2d4bdd0af534bf8b270644df2f98eaf, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family2/testCompactionWithHFileLink=daa57ebba7c140c319a2f6facc725e4a-13ca0708b05643899307a03f0fcc261a, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family2/testCompactionWithHFileLink=daa57ebba7c140c319a2f6facc725e4a-9173733bfa77411fa7038a412b2f97e1] into tmpdir=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/.tmp, totalSize=20.4 K 2023-03-05 19:55:43,259 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=102, resume processing ppid=98 2023-03-05 19:55:43,259 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=102, ppid=98, state=SUCCESS; OpenRegionProcedure daa57ebba7c140c319a2f6facc725e4a, server=jenkins-hbase4.apache.org,37037,1678046018983 in 219 msec 2023-03-05 19:55:43,259 DEBUG [StoreOpener-942afd2d0a4ceef025fd5a607956d53d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family1 2023-03-05 19:55:43,260 DEBUG [StoreOpener-942afd2d0a4ceef025fd5a607956d53d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family1 2023-03-05 19:55:43,260 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=daa57ebba7c140c319a2f6facc725e4a-f6b69932b5da4e0d9155ef905eacf40c, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678046119974 2023-03-05 19:55:43,260 DEBUG [RS:0;jenkins-hbase4:37037-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=daa57ebba7c140c319a2f6facc725e4a-c42309b45f3243f2a021144945ad30f4, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678046118195 2023-03-05 19:55:43,260 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=98, ppid=97, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=daa57ebba7c140c319a2f6facc725e4a, ASSIGN in 377 msec 2023-03-05 19:55:43,260 INFO [StoreOpener-942afd2d0a4ceef025fd5a607956d53d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 942afd2d0a4ceef025fd5a607956d53d columnFamilyName family1 2023-03-05 19:55:43,260 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=daa57ebba7c140c319a2f6facc725e4a-43db0ea3bd404e2fa10949ad9df289b4, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678046121773 2023-03-05 19:55:43,261 DEBUG [RS:0;jenkins-hbase4:37037-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=daa57ebba7c140c319a2f6facc725e4a-c2d4bdd0af534bf8b270644df2f98eaf, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678046119974 2023-03-05 19:55:43,261 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=daa57ebba7c140c319a2f6facc725e4a-e02bc7cebc7e44af831019cee45cd29b, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678046123595 2023-03-05 19:55:43,262 DEBUG [RS:0;jenkins-hbase4:37037-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=daa57ebba7c140c319a2f6facc725e4a-13ca0708b05643899307a03f0fcc261a, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678046121773 2023-03-05 19:55:43,263 DEBUG [RS:0;jenkins-hbase4:37037-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=daa57ebba7c140c319a2f6facc725e4a-9173733bfa77411fa7038a412b2f97e1, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678046123595 2023-03-05 19:55:43,275 DEBUG [StoreOpener-942afd2d0a4ceef025fd5a607956d53d-1] regionserver.HStore(539): loaded hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family1/testCompactionWithHFileLink=942afd2d0a4ceef025fd5a607956d53d-2ddb935fe9dc46778a4c450ff10d3c44 2023-03-05 19:55:43,284 DEBUG [StoreOpener-942afd2d0a4ceef025fd5a607956d53d-1] regionserver.HStore(539): loaded hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family1/testCompactionWithHFileLink=942afd2d0a4ceef025fd5a607956d53d-8139f54670e14aa3a572f9bdb2477108 2023-03-05 19:55:43,299 DEBUG [StoreOpener-942afd2d0a4ceef025fd5a607956d53d-1] regionserver.HStore(539): loaded hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family1/testCompactionWithHFileLink=942afd2d0a4ceef025fd5a607956d53d-c88f1bd396f4409ca38794d504f5c156 2023-03-05 19:55:43,300 INFO [RS:0;jenkins-hbase4:37037-shortCompactions-0] throttle.PressureAwareThroughputController(145): daa57ebba7c140c319a2f6facc725e4a#family1#compaction#113 average throughput is 0.26 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-03-05 19:55:43,308 INFO [RS:0;jenkins-hbase4:37037-longCompactions-0] throttle.PressureAwareThroughputController(145): daa57ebba7c140c319a2f6facc725e4a#family2#compaction#114 average throughput is 0.10 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-05 19:55:43,312 DEBUG [StoreOpener-942afd2d0a4ceef025fd5a607956d53d-1] regionserver.HStore(539): loaded hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family1/testCompactionWithHFileLink=942afd2d0a4ceef025fd5a607956d53d-dc8108fdcc6a4a0a9720b3e2336cb620 2023-03-05 19:55:43,316 INFO [StoreOpener-942afd2d0a4ceef025fd5a607956d53d-1] regionserver.HStore(310): Store=942afd2d0a4ceef025fd5a607956d53d/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-05 19:55:43,316 INFO [StoreOpener-942afd2d0a4ceef025fd5a607956d53d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 942afd2d0a4ceef025fd5a607956d53d 2023-03-05 19:55:43,317 DEBUG [StoreOpener-942afd2d0a4ceef025fd5a607956d53d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family2 2023-03-05 19:55:43,317 DEBUG [StoreOpener-942afd2d0a4ceef025fd5a607956d53d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family2 2023-03-05 19:55:43,318 INFO [StoreOpener-942afd2d0a4ceef025fd5a607956d53d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 942afd2d0a4ceef025fd5a607956d53d columnFamilyName family2 2023-03-05 19:55:43,331 DEBUG [StoreOpener-942afd2d0a4ceef025fd5a607956d53d-1] regionserver.HStore(539): loaded hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family2/testCompactionWithHFileLink=942afd2d0a4ceef025fd5a607956d53d-2c3de0e4dce5444ca51da5f66eaf06a1 2023-03-05 19:55:43,334 DEBUG [StoreOpener-942afd2d0a4ceef025fd5a607956d53d-1] regionserver.HStore(539): loaded hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family2/testCompactionWithHFileLink=942afd2d0a4ceef025fd5a607956d53d-55d239cb7531440088e28c0c9e6143d3 2023-03-05 19:55:43,338 DEBUG [StoreOpener-942afd2d0a4ceef025fd5a607956d53d-1] regionserver.HStore(539): loaded hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family2/testCompactionWithHFileLink=942afd2d0a4ceef025fd5a607956d53d-9a13755a01ce4e01b7e321e95b43b8cc 2023-03-05 19:55:43,340 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/.tmp/family1/55510d9acc0d4855893bfcb22be98340 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family1/55510d9acc0d4855893bfcb22be98340 2023-03-05 19:55:43,342 DEBUG [StoreOpener-942afd2d0a4ceef025fd5a607956d53d-1] regionserver.HStore(539): loaded hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family2/testCompactionWithHFileLink=942afd2d0a4ceef025fd5a607956d53d-af93404d81154179a4c27fce0e335552 2023-03-05 19:55:43,342 INFO [StoreOpener-942afd2d0a4ceef025fd5a607956d53d-1] regionserver.HStore(310): Store=942afd2d0a4ceef025fd5a607956d53d/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-05 19:55:43,343 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d 2023-03-05 19:55:43,344 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d 2023-03-05 19:55:43,345 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testCompactionWithHFileLink' 2023-03-05 19:55:43,345 INFO [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in daa57ebba7c140c319a2f6facc725e4a/family1 of daa57ebba7c140c319a2f6facc725e4a into 55510d9acc0d4855893bfcb22be98340(size=7.6 K), total size for store is 7.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-05 19:55:43,345 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for daa57ebba7c140c319a2f6facc725e4a: 2023-03-05 19:55:43,345 INFO [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,,1678046116082.daa57ebba7c140c319a2f6facc725e4a., storeName=daa57ebba7c140c319a2f6facc725e4a/family1, priority=12, startTime=1678046143253; duration=0sec 2023-03-05 19:55:43,345 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-05 19:55:43,346 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-05 19:55:43,348 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 942afd2d0a4ceef025fd5a607956d53d 2023-03-05 19:55:43,348 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 942afd2d0a4ceef025fd5a607956d53d; next sequenceid=32; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71498246, jitterRate=0.06540688872337341}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-05 19:55:43,348 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 942afd2d0a4ceef025fd5a607956d53d: 2023-03-05 19:55:43,349 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,1,1678046116082.942afd2d0a4ceef025fd5a607956d53d., pid=103, masterSystemTime=1678046143189 2023-03-05 19:55:43,349 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-05 19:55:43,350 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-03-05 19:55:43,350 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-03-05 19:55:43,351 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,1,1678046116082.942afd2d0a4ceef025fd5a607956d53d. 2023-03-05 19:55:43,351 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,1,1678046116082.942afd2d0a4ceef025fd5a607956d53d. 2023-03-05 19:55:43,351 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,2,1678046116082.25af79a83c9b248c956b57fc8507ead6. 2023-03-05 19:55:43,351 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 25af79a83c9b248c956b57fc8507ead6, NAME => 'testCompactionWithHFileLink,2,1678046116082.25af79a83c9b248c956b57fc8507ead6.', STARTKEY => '2', ENDKEY => ''} 2023-03-05 19:55:43,352 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=99 updating hbase:meta row=942afd2d0a4ceef025fd5a607956d53d, regionState=OPEN, openSeqNum=32, regionLocation=jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:43,352 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink 25af79a83c9b248c956b57fc8507ead6 2023-03-05 19:55:43,352 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 22152 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-03-05 19:55:43,352 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,1,1678046116082.942afd2d0a4ceef025fd5a607956d53d.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1678046143352"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678046143352"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678046143352"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678046143352"}]},"ts":"1678046143352"} 2023-03-05 19:55:43,352 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.HStore(1912): 942afd2d0a4ceef025fd5a607956d53d/family1 is initiating minor compaction (all files) 2023-03-05 19:55:43,352 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,2,1678046116082.25af79a83c9b248c956b57fc8507ead6.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-05 19:55:43,352 INFO [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 942afd2d0a4ceef025fd5a607956d53d/family1 in testCompactionWithHFileLink,1,1678046116082.942afd2d0a4ceef025fd5a607956d53d. 2023-03-05 19:55:43,352 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 25af79a83c9b248c956b57fc8507ead6 2023-03-05 19:55:43,352 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 25af79a83c9b248c956b57fc8507ead6 2023-03-05 19:55:43,352 INFO [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family1/testCompactionWithHFileLink=942afd2d0a4ceef025fd5a607956d53d-2ddb935fe9dc46778a4c450ff10d3c44, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family1/testCompactionWithHFileLink=942afd2d0a4ceef025fd5a607956d53d-dc8108fdcc6a4a0a9720b3e2336cb620, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family1/testCompactionWithHFileLink=942afd2d0a4ceef025fd5a607956d53d-c88f1bd396f4409ca38794d504f5c156, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family1/testCompactionWithHFileLink=942afd2d0a4ceef025fd5a607956d53d-8139f54670e14aa3a572f9bdb2477108] into tmpdir=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/.tmp, totalSize=21.6 K 2023-03-05 19:55:43,353 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=942afd2d0a4ceef025fd5a607956d53d-2ddb935fe9dc46778a4c450ff10d3c44, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678046118197 2023-03-05 19:55:43,353 INFO [StoreOpener-25af79a83c9b248c956b57fc8507ead6-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 25af79a83c9b248c956b57fc8507ead6 2023-03-05 19:55:43,353 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=942afd2d0a4ceef025fd5a607956d53d-dc8108fdcc6a4a0a9720b3e2336cb620, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678046119974 2023-03-05 19:55:43,354 DEBUG [StoreOpener-25af79a83c9b248c956b57fc8507ead6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family1 2023-03-05 19:55:43,354 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=942afd2d0a4ceef025fd5a607956d53d-c88f1bd396f4409ca38794d504f5c156, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678046121773 2023-03-05 19:55:43,354 DEBUG [StoreOpener-25af79a83c9b248c956b57fc8507ead6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family1 2023-03-05 19:55:43,354 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=103, resume processing ppid=99 2023-03-05 19:55:43,354 INFO [StoreOpener-25af79a83c9b248c956b57fc8507ead6-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 25af79a83c9b248c956b57fc8507ead6 columnFamilyName family1 2023-03-05 19:55:43,355 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=103, ppid=99, state=SUCCESS; OpenRegionProcedure 942afd2d0a4ceef025fd5a607956d53d, server=jenkins-hbase4.apache.org,37037,1678046018983 in 314 msec 2023-03-05 19:55:43,355 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=942afd2d0a4ceef025fd5a607956d53d-8139f54670e14aa3a572f9bdb2477108, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678046123595 2023-03-05 19:55:43,355 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=99, ppid=97, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=942afd2d0a4ceef025fd5a607956d53d, ASSIGN in 472 msec 2023-03-05 19:55:43,364 DEBUG [StoreOpener-25af79a83c9b248c956b57fc8507ead6-1] regionserver.HStore(539): loaded hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family1/testCompactionWithHFileLink=25af79a83c9b248c956b57fc8507ead6-24f57a619907417b8e8a0222c800bc07 2023-03-05 19:55:43,369 DEBUG [StoreOpener-25af79a83c9b248c956b57fc8507ead6-1] regionserver.HStore(539): loaded hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family1/testCompactionWithHFileLink=25af79a83c9b248c956b57fc8507ead6-3004199e82364ee9abaf18f0f843fc76 2023-03-05 19:55:43,375 DEBUG [StoreOpener-25af79a83c9b248c956b57fc8507ead6-1] regionserver.HStore(539): loaded hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family1/testCompactionWithHFileLink=25af79a83c9b248c956b57fc8507ead6-6b8608e70a864d6e8a4e55c85261adfe 2023-03-05 19:55:43,384 DEBUG [StoreOpener-25af79a83c9b248c956b57fc8507ead6-1] regionserver.HStore(539): loaded hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family1/testCompactionWithHFileLink=25af79a83c9b248c956b57fc8507ead6-9a9155a47d074f5f9005ddf6337e173e 2023-03-05 19:55:43,385 INFO [StoreOpener-25af79a83c9b248c956b57fc8507ead6-1] regionserver.HStore(310): Store=25af79a83c9b248c956b57fc8507ead6/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-05 19:55:43,385 INFO [RS:0;jenkins-hbase4:37037-shortCompactions-0] throttle.PressureAwareThroughputController(145): 942afd2d0a4ceef025fd5a607956d53d#family1#compaction#115 average throughput is 0.47 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-05 19:55:43,385 INFO [StoreOpener-25af79a83c9b248c956b57fc8507ead6-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 25af79a83c9b248c956b57fc8507ead6 2023-03-05 19:55:43,385 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family1/testCompactionWithHFileLink=daa57ebba7c140c319a2f6facc725e4a-cdcb9dd831794696af2aaf30754840ee, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family1/testCompactionWithHFileLink=daa57ebba7c140c319a2f6facc725e4a-f6b69932b5da4e0d9155ef905eacf40c, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family1/testCompactionWithHFileLink=daa57ebba7c140c319a2f6facc725e4a-43db0ea3bd404e2fa10949ad9df289b4, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family1/testCompactionWithHFileLink=daa57ebba7c140c319a2f6facc725e4a-e02bc7cebc7e44af831019cee45cd29b] to archive 2023-03-05 19:55:43,386 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-05 19:55:43,386 DEBUG [StoreOpener-25af79a83c9b248c956b57fc8507ead6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family2 2023-03-05 19:55:43,386 DEBUG [StoreOpener-25af79a83c9b248c956b57fc8507ead6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family2 2023-03-05 19:55:43,386 INFO [StoreOpener-25af79a83c9b248c956b57fc8507ead6-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 25af79a83c9b248c956b57fc8507ead6 columnFamilyName family2 2023-03-05 19:55:43,387 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family1/testCompactionWithHFileLink=daa57ebba7c140c319a2f6facc725e4a-cdcb9dd831794696af2aaf30754840ee to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family1/testCompactionWithHFileLink=daa57ebba7c140c319a2f6facc725e4a-cdcb9dd831794696af2aaf30754840ee 2023-03-05 19:55:43,388 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family1/testCompactionWithHFileLink=daa57ebba7c140c319a2f6facc725e4a-f6b69932b5da4e0d9155ef905eacf40c to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family1/testCompactionWithHFileLink=daa57ebba7c140c319a2f6facc725e4a-f6b69932b5da4e0d9155ef905eacf40c 2023-03-05 19:55:43,392 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family1/testCompactionWithHFileLink=daa57ebba7c140c319a2f6facc725e4a-43db0ea3bd404e2fa10949ad9df289b4 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family1/testCompactionWithHFileLink=daa57ebba7c140c319a2f6facc725e4a-43db0ea3bd404e2fa10949ad9df289b4 2023-03-05 19:55:43,393 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family1/testCompactionWithHFileLink=daa57ebba7c140c319a2f6facc725e4a-e02bc7cebc7e44af831019cee45cd29b to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family1/testCompactionWithHFileLink=daa57ebba7c140c319a2f6facc725e4a-e02bc7cebc7e44af831019cee45cd29b 2023-03-05 19:55:43,395 DEBUG [StoreOpener-25af79a83c9b248c956b57fc8507ead6-1] regionserver.HStore(539): loaded hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family2/testCompactionWithHFileLink=25af79a83c9b248c956b57fc8507ead6-11fd411913d44c5187ed3f273ed56bdd 2023-03-05 19:55:43,399 DEBUG [StoreOpener-25af79a83c9b248c956b57fc8507ead6-1] regionserver.HStore(539): loaded hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family2/testCompactionWithHFileLink=25af79a83c9b248c956b57fc8507ead6-1dfce094a6164169b84ae1182f0260e1 2023-03-05 19:55:43,402 DEBUG [StoreOpener-25af79a83c9b248c956b57fc8507ead6-1] regionserver.HStore(539): loaded hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family2/testCompactionWithHFileLink=25af79a83c9b248c956b57fc8507ead6-22cbfc0fa2af442f976458cd740e398d 2023-03-05 19:55:43,406 DEBUG [StoreOpener-25af79a83c9b248c956b57fc8507ead6-1] regionserver.HStore(539): loaded hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family2/testCompactionWithHFileLink=25af79a83c9b248c956b57fc8507ead6-e2b047fb6d65462d939ae96ac7375619 2023-03-05 19:55:43,406 INFO [StoreOpener-25af79a83c9b248c956b57fc8507ead6-1] regionserver.HStore(310): Store=25af79a83c9b248c956b57fc8507ead6/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-05 19:55:43,407 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6 2023-03-05 19:55:43,408 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6 2023-03-05 19:55:43,409 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-05 19:55:43,410 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 25af79a83c9b248c956b57fc8507ead6 2023-03-05 19:55:43,411 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 25af79a83c9b248c956b57fc8507ead6; next sequenceid=32; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71621511, jitterRate=0.06724368035793304}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-05 19:55:43,411 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 25af79a83c9b248c956b57fc8507ead6: 2023-03-05 19:55:43,411 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,2,1678046116082.25af79a83c9b248c956b57fc8507ead6., pid=101, masterSystemTime=1678046143189 2023-03-05 19:55:43,411 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-03-05 19:55:43,411 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=3), splitQueue=0 2023-03-05 19:55:43,412 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,2,1678046116082.25af79a83c9b248c956b57fc8507ead6. 2023-03-05 19:55:43,413 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,2,1678046116082.25af79a83c9b248c956b57fc8507ead6. 2023-03-05 19:55:43,413 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=100 updating hbase:meta row=25af79a83c9b248c956b57fc8507ead6, regionState=OPEN, openSeqNum=32, regionLocation=jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:43,413 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,2,1678046116082.25af79a83c9b248c956b57fc8507ead6.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678046143413"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678046143413"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678046143413"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678046143413"}]},"ts":"1678046143413"} 2023-03-05 19:55:43,415 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=101, resume processing ppid=100 2023-03-05 19:55:43,415 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=101, ppid=100, state=SUCCESS; OpenRegionProcedure 25af79a83c9b248c956b57fc8507ead6, server=jenkins-hbase4.apache.org,37037,1678046018983 in 376 msec 2023-03-05 19:55:43,416 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=100, resume processing ppid=97 2023-03-05 19:55:43,416 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=100, ppid=97, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=25af79a83c9b248c956b57fc8507ead6, ASSIGN in 533 msec 2023-03-05 19:55:43,417 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678046143417"}]},"ts":"1678046143417"} 2023-03-05 19:55:43,418 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=ENABLED in hbase:meta 2023-03-05 19:55:43,419 INFO [PEWorker-3] procedure.EnableTableProcedure(310): Table 'testCompactionWithHFileLink' was successfully enabled. 2023-03-05 19:55:43,420 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=97, state=SUCCESS; EnableTableProcedure table=testCompactionWithHFileLink in 548 msec 2023-03-05 19:55:43,481 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1227): Checking to see if procedure is done pid=97 2023-03-05 19:55:43,482 INFO [Listener at localhost/40787] client.HBaseAdmin$TableFuture(3541): Operation: ENABLE, Table Name: default:testCompactionWithHFileLink, procId: 97 completed 2023-03-05 19:55:43,489 DEBUG [Listener at localhost/40787] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-05 19:55:43,500 DEBUG [Listener at localhost/40787] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-05 19:55:43,508 DEBUG [Listener at localhost/40787] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-05 19:55:43,508 DEBUG [Listener at localhost/40787] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-05 19:55:43,513 INFO [Listener at localhost/40787] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-03-05 19:55:43,525 INFO [Listener at localhost/40787] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 12 selected files, and 0 irrelevant files table 'testCompactionWithHFileLink' and column 'family1' 2023-03-05 19:55:43,525 INFO [Listener at localhost/40787] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testCompactionWithHFileLink' column='family1' 2023-03-05 19:55:43,525 DEBUG [Listener at localhost/40787] zookeeper.ReadOnlyZKClient(139): Connect 0x096139a4 to 127.0.0.1:62957 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-05 19:55:43,529 DEBUG [Listener at localhost/40787] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7e586a95, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-05 19:55:43,529 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230305 for table='testCompactionWithHFileLink' and column='family1' 2023-03-05 19:55:43,529 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c4ca4238a0b923820dcc509a6f75849b20230305 for table='testCompactionWithHFileLink' and column='family1' 2023-03-05 19:55:43,529 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230305 for table='testCompactionWithHFileLink' and column='family1' 2023-03-05 19:55:43,998 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230305863b97e6d0c04b158d0d40026d779d9f to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230305863b97e6d0c04b158d0d40026d779d9f 2023-03-05 19:55:43,998 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303056b6d9c0d31f34a1997348707c824e605 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202303056b6d9c0d31f34a1997348707c824e605 2023-03-05 19:55:43,999 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230305 table='testCompactionWithHFileLink' column='family1' 2023-03-05 19:55:43,999 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230305 table='testCompactionWithHFileLink' column='family1' 2023-03-05 19:55:44,001 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023030578c1350f9d174781b9fa936774bf6098 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023030578c1350f9d174781b9fa936774bf6098 2023-03-05 19:55:44,002 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230305 table='testCompactionWithHFileLink' column='family1' 2023-03-05 19:55:44,012 DEBUG [MobFileCompactionChore-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-05 19:55:44,015 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:43264, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-05 19:55:44,022 DEBUG [MobFileCompactionChore-pool-2] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-05 19:55:44,024 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:46096, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-05 19:55:44,046 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230305/family1/9c8345e74a62455abf23c07a19935781 first=Optional[0\x00\x00\x00\x00] last=Optional[0\x00\x00\x00\x07] 2023-03-05 19:55:44,046 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230305/family1/6b1d1509ef814d168b2b0d744b5ed4f6 first=Optional[1\x00\x00\x00\x00] last=Optional[1\x00\x00\x00\x07] 2023-03-05 19:55:44,048 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testCompactionWithHFileLink,,1678046116082.daa57ebba7c140c319a2f6facc725e4a., hostname=jenkins-hbase4.apache.org,37037,1678046018983, seqNum=32 for row with hfile group [{family1,hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230305/family1/9c8345e74a62455abf23c07a19935781}] 2023-03-05 19:55:44,048 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testCompactionWithHFileLink,1,1678046116082.942afd2d0a4ceef025fd5a607956d53d., hostname=jenkins-hbase4.apache.org,37037,1678046018983, seqNum=32 for row 1 with hfile group [{family1,hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230305/family1/6b1d1509ef814d168b2b0d744b5ed4f6}] 2023-03-05 19:55:44,049 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230305/family1/abe71a5484b24d03bf3143365292b640 first=Optional[2\x00\x00\x00\x00] last=Optional[2\x00\x00\x00\x07] 2023-03-05 19:55:44,050 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testCompactionWithHFileLink,2,1678046116082.25af79a83c9b248c956b57fc8507ead6., hostname=jenkins-hbase4.apache.org,37037,1678046018983, seqNum=32 for row 2 with hfile group [{family1,hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230305/family1/abe71a5484b24d03bf3143365292b640}] 2023-03-05 19:55:44,057 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37037] regionserver.HStore(763): Validating hfile at hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230305/family1/6b1d1509ef814d168b2b0d744b5ed4f6 for inclusion in 942afd2d0a4ceef025fd5a607956d53d/family1 2023-03-05 19:55:44,057 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.HStore(763): Validating hfile at hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230305/family1/abe71a5484b24d03bf3143365292b640 for inclusion in 25af79a83c9b248c956b57fc8507ead6/family1 2023-03-05 19:55:44,057 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(763): Validating hfile at hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230305/family1/9c8345e74a62455abf23c07a19935781 for inclusion in daa57ebba7c140c319a2f6facc725e4a/family1 2023-03-05 19:55:44,060 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.HStore(775): HFile bounds: first=2\x00\x00\x00\x00 last=2\x00\x00\x00\x07 2023-03-05 19:55:44,060 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37037] regionserver.HStore(775): HFile bounds: first=1\x00\x00\x00\x00 last=1\x00\x00\x00\x07 2023-03-05 19:55:44,060 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37037] regionserver.HStore(777): Region bounds: first=1 last=2 2023-03-05 19:55:44,060 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.HStore(777): Region bounds: first=2 last= 2023-03-05 19:55:44,060 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37037] regionserver.HRegion(2446): Flush status journal for 942afd2d0a4ceef025fd5a607956d53d: 2023-03-05 19:55:44,061 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.HRegion(2446): Flush status journal for 25af79a83c9b248c956b57fc8507ead6: 2023-03-05 19:55:44,062 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(775): HFile bounds: first=0\x00\x00\x00\x00 last=0\x00\x00\x00\x07 2023-03-05 19:55:44,062 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(777): Region bounds: first= last=1 2023-03-05 19:55:44,062 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HRegion(2446): Flush status journal for daa57ebba7c140c319a2f6facc725e4a: 2023-03-05 19:55:44,063 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37037] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230305/family1/6b1d1509ef814d168b2b0d744b5ed4f6 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testCompactionWithHFileLink__m7p0dp433oiuhsr7vigsipo8s75mpeelass888jn4vnmfr2s22tkldvj3vfdfps/family1/6b1d1509ef814d168b2b0d744b5ed4f6 2023-03-05 19:55:44,063 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230305/family1/abe71a5484b24d03bf3143365292b640 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testCompactionWithHFileLink__ftku8cpkr0v4khqm74ard4gnqklaaga13h2d1t2il39uu1sd8avbr7058bbbvia5/family1/abe71a5484b24d03bf3143365292b640 2023-03-05 19:55:44,063 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230305/family1/9c8345e74a62455abf23c07a19935781 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testCompactionWithHFileLink__1st7e4c7qrasjai4c5ivhmacb7cc2q0fho65ondd4ei245ncq86ojsm1tgqvl2fd/family1/9c8345e74a62455abf23c07a19935781 2023-03-05 19:55:44,065 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37037] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testCompactionWithHFileLink__m7p0dp433oiuhsr7vigsipo8s75mpeelass888jn4vnmfr2s22tkldvj3vfdfps/family1/6b1d1509ef814d168b2b0d744b5ed4f6 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family1/96acc36d509049e7b8f3e1ffd14db09b_SeqId_34_ 2023-03-05 19:55:44,066 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testCompactionWithHFileLink__ftku8cpkr0v4khqm74ard4gnqklaaga13h2d1t2il39uu1sd8avbr7058bbbvia5/family1/abe71a5484b24d03bf3143365292b640 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family1/2a347452bcd24b71a1bd7bdd97a27786_SeqId_34_ 2023-03-05 19:55:44,066 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testCompactionWithHFileLink__1st7e4c7qrasjai4c5ivhmacb7cc2q0fho65ondd4ei245ncq86ojsm1tgqvl2fd/family1/9c8345e74a62455abf23c07a19935781 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family1/5234eb4046424b278362f9d85801d4b9_SeqId_35_ 2023-03-05 19:55:44,066 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37037] regionserver.HStore(849): Loaded HFile hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testCompactionWithHFileLink__m7p0dp433oiuhsr7vigsipo8s75mpeelass888jn4vnmfr2s22tkldvj3vfdfps/family1/6b1d1509ef814d168b2b0d744b5ed4f6 into 942afd2d0a4ceef025fd5a607956d53d/family1 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family1/96acc36d509049e7b8f3e1ffd14db09b_SeqId_34_ - updating store file list. 2023-03-05 19:55:44,066 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.HStore(849): Loaded HFile hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testCompactionWithHFileLink__ftku8cpkr0v4khqm74ard4gnqklaaga13h2d1t2il39uu1sd8avbr7058bbbvia5/family1/abe71a5484b24d03bf3143365292b640 into 25af79a83c9b248c956b57fc8507ead6/family1 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family1/2a347452bcd24b71a1bd7bdd97a27786_SeqId_34_ - updating store file list. 2023-03-05 19:55:44,067 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(849): Loaded HFile hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testCompactionWithHFileLink__1st7e4c7qrasjai4c5ivhmacb7cc2q0fho65ondd4ei245ncq86ojsm1tgqvl2fd/family1/9c8345e74a62455abf23c07a19935781 into daa57ebba7c140c319a2f6facc725e4a/family1 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family1/5234eb4046424b278362f9d85801d4b9_SeqId_35_ - updating store file list. 2023-03-05 19:55:44,070 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37037] regionserver.HStore(882): Loaded HFile hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family1/96acc36d509049e7b8f3e1ffd14db09b_SeqId_34_ into 942afd2d0a4ceef025fd5a607956d53d/family1 2023-03-05 19:55:44,070 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37037] regionserver.HStore(855): Successfully loaded hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testCompactionWithHFileLink__m7p0dp433oiuhsr7vigsipo8s75mpeelass888jn4vnmfr2s22tkldvj3vfdfps/family1/6b1d1509ef814d168b2b0d744b5ed4f6 into 942afd2d0a4ceef025fd5a607956d53d/family1 (new location: hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family1/96acc36d509049e7b8f3e1ffd14db09b_SeqId_34_) 2023-03-05 19:55:44,070 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.HStore(882): Loaded HFile hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family1/2a347452bcd24b71a1bd7bdd97a27786_SeqId_34_ into 25af79a83c9b248c956b57fc8507ead6/family1 2023-03-05 19:55:44,070 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.HStore(855): Successfully loaded hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testCompactionWithHFileLink__ftku8cpkr0v4khqm74ard4gnqklaaga13h2d1t2il39uu1sd8avbr7058bbbvia5/family1/abe71a5484b24d03bf3143365292b640 into 25af79a83c9b248c956b57fc8507ead6/family1 (new location: hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family1/2a347452bcd24b71a1bd7bdd97a27786_SeqId_34_) 2023-03-05 19:55:44,070 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37037] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testCompactionWithHFileLink__m7p0dp433oiuhsr7vigsipo8s75mpeelass888jn4vnmfr2s22tkldvj3vfdfps/family1/6b1d1509ef814d168b2b0d744b5ed4f6 2023-03-05 19:55:44,071 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(882): Loaded HFile hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family1/5234eb4046424b278362f9d85801d4b9_SeqId_35_ into daa57ebba7c140c319a2f6facc725e4a/family1 2023-03-05 19:55:44,071 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.HStore(855): Successfully loaded hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testCompactionWithHFileLink__1st7e4c7qrasjai4c5ivhmacb7cc2q0fho65ondd4ei245ncq86ojsm1tgqvl2fd/family1/9c8345e74a62455abf23c07a19935781 into daa57ebba7c140c319a2f6facc725e4a/family1 (new location: hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family1/5234eb4046424b278362f9d85801d4b9_SeqId_35_) 2023-03-05 19:55:44,071 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testCompactionWithHFileLink__ftku8cpkr0v4khqm74ard4gnqklaaga13h2d1t2il39uu1sd8avbr7058bbbvia5/family1/abe71a5484b24d03bf3143365292b640 2023-03-05 19:55:44,071 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-05 19:55:44,071 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37037] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testCompactionWithHFileLink__1st7e4c7qrasjai4c5ivhmacb7cc2q0fho65ondd4ei245ncq86ojsm1tgqvl2fd/family1/9c8345e74a62455abf23c07a19935781 2023-03-05 19:55:44,072 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230305 table='testCompactionWithHFileLink' column='family1' 2023-03-05 19:55:44,073 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230305 table='testCompactionWithHFileLink' column='family1' 2023-03-05 19:55:44,073 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-05 19:55:44,073 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(360): Archiving compacted files. 2023-03-05 19:55:44,074 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 5 files of size 29606 starting at candidate #0 after considering 6 permutations with 6 in ratio 2023-03-05 19:55:44,074 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.HStore(1912): 25af79a83c9b248c956b57fc8507ead6/family1 is initiating minor compaction (all files) 2023-03-05 19:55:44,074 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@2907f7d; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=4), splitQueue=0 2023-03-05 19:55:44,074 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b2023030501cbec120485489fa59d0ed556ae93f6 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b2023030501cbec120485489fa59d0ed556ae93f6 2023-03-05 19:55:44,075 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e202303056a09d4fe8b054b179c0313b89fbc21ce to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e202303056a09d4fe8b054b179c0313b89fbc21ce 2023-03-05 19:55:44,076 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230305 table='testCompactionWithHFileLink' column='family1' 2023-03-05 19:55:44,076 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b20230305202e97351c394119851f4075ef789695 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b20230305202e97351c394119851f4075ef789695 2023-03-05 19:55:44,076 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20230305a33235cee5224842811abef82371aea6 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20230305a33235cee5224842811abef82371aea6 2023-03-05 19:55:44,076 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(360): Archiving compacted files. 2023-03-05 19:55:44,077 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b202303058002b27e00b04b36af1ade5009390107 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b202303058002b27e00b04b36af1ade5009390107 2023-03-05 19:55:44,077 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20230305da73569395114dad837fdbde0dcbba50 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20230305da73569395114dad837fdbde0dcbba50 2023-03-05 19:55:44,077 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c2023030524196f3846324173b9e8c56ba282d604 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c2023030524196f3846324173b9e8c56ba282d604 2023-03-05 19:55:44,078 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b20230305fae8652a174c472c8be7df36b3f66be3 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b20230305fae8652a174c472c8be7df36b3f66be3 2023-03-05 19:55:44,078 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-03-05 19:55:44,078 DEBUG [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230305' 2023-03-05 19:55:44,078 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20230305f43d564866564d9e939ce4a0e9ba6b2f to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20230305f43d564866564d9e939ce4a0e9ba6b2f 2023-03-05 19:55:44,078 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-03-05 19:55:44,078 DEBUG [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230305' 2023-03-05 19:55:44,078 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c4ca4238a0b923820dcc509a6f75849b20230305 for table='testCompactionWithHFileLink' and column='family1' 2023-03-05 19:55:44,078 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c202303056d7220390ee448d6a463d2fc4c42f4da to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c202303056d7220390ee448d6a463d2fc4c42f4da 2023-03-05 19:55:44,078 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230305 for table='testCompactionWithHFileLink' and column='family1' 2023-03-05 19:55:44,079 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c202303057212e8b9c498437fb3c20726fe55a714 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c202303057212e8b9c498437fb3c20726fe55a714 2023-03-05 19:55:44,080 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c202303058825f16fb4854d6eb896c400e4d85db8 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c202303058825f16fb4854d6eb896c400e4d85db8 2023-03-05 19:55:44,080 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-03-05 19:55:44,080 DEBUG [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230305' 2023-03-05 19:55:44,080 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230305 for table='testCompactionWithHFileLink' and column='family1' 2023-03-05 19:55:44,080 INFO [Listener at localhost/40787] compactions.PartitionedMobCompactor(354): After compaction, there are 3 mob files. table='testCompactionWithHFileLink' column='family1' 2023-03-05 19:55:44,082 DEBUG [Listener at localhost/40787] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-05 19:55:44,084 DEBUG [Listener at localhost/40787] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-05 19:55:44,086 DEBUG [Listener at localhost/40787] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-05 19:55:44,087 DEBUG [Listener at localhost/40787] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-05 19:55:44,093 DEBUG [Listener at localhost/40787] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-05 19:55:44,093 DEBUG [Listener at localhost/40787] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-05 19:55:44,109 INFO [Listener at localhost/40787] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testCompactionWithHFileLink Thread=289 (was 244) - Thread LEAK? -, OpenFileDescriptor=760 (was 616) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=90 (was 140), ProcessCount=171 (was 173), AvailableMemoryMB=4574 (was 4665) 2023-03-05 19:55:44,121 INFO [Listener at localhost/40787] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMinorCompaction Thread=289, OpenFileDescriptor=760, MaxFileDescriptor=60000, SystemLoadAverage=90, ProcessCount=171, AvailableMemoryMB=4573 2023-03-05 19:55:44,126 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.HMaster$15(3014): Client=jenkins//172.31.14.131 creating {NAME => 'ns'} 2023-03-05 19:55:44,127 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure2.ProcedureExecutor(1029): Stored pid=104, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=ns 2023-03-05 19:55:44,131 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1227): Checking to see if procedure is done pid=104 2023-03-05 19:55:44,133 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-03-05 19:55:44,135 DEBUG [RS:0;jenkins-hbase4:37037-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/.tmp/family2/a6c02da481f44160a7aa54bb97b42eae as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family2/a6c02da481f44160a7aa54bb97b42eae 2023-03-05 19:55:44,135 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=104, state=SUCCESS; CreateNamespaceProcedure, namespace=ns in 8 msec 2023-03-05 19:55:44,140 INFO [RS:0;jenkins-hbase4:37037-longCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in daa57ebba7c140c319a2f6facc725e4a/family2 of daa57ebba7c140c319a2f6facc725e4a into a6c02da481f44160a7aa54bb97b42eae(size=6.4 K), total size for store is 6.4 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-05 19:55:44,140 DEBUG [RS:0;jenkins-hbase4:37037-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for daa57ebba7c140c319a2f6facc725e4a: 2023-03-05 19:55:44,140 INFO [RS:0;jenkins-hbase4:37037-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,,1678046116082.daa57ebba7c140c319a2f6facc725e4a., storeName=daa57ebba7c140c319a2f6facc725e4a/family2, priority=12, startTime=1678046143254; duration=0sec 2023-03-05 19:55:44,140 DEBUG [RS:0;jenkins-hbase4:37037-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=4), splitQueue=0 2023-03-05 19:55:44,140 INFO [RS:0;jenkins-hbase4:37037-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 25af79a83c9b248c956b57fc8507ead6/family1 in testCompactionWithHFileLink,2,1678046116082.25af79a83c9b248c956b57fc8507ead6. 2023-03-05 19:55:44,140 INFO [RS:0;jenkins-hbase4:37037-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family1/testCompactionWithHFileLink=25af79a83c9b248c956b57fc8507ead6-9a9155a47d074f5f9005ddf6337e173e, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family1/testCompactionWithHFileLink=25af79a83c9b248c956b57fc8507ead6-24f57a619907417b8e8a0222c800bc07, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family1/testCompactionWithHFileLink=25af79a83c9b248c956b57fc8507ead6-3004199e82364ee9abaf18f0f843fc76, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family1/testCompactionWithHFileLink=25af79a83c9b248c956b57fc8507ead6-6b8608e70a864d6e8a4e55c85261adfe, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family1/2a347452bcd24b71a1bd7bdd97a27786_SeqId_34_] into tmpdir=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/.tmp, totalSize=28.9 K 2023-03-05 19:55:44,140 DEBUG [RS:0;jenkins-hbase4:37037-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=25af79a83c9b248c956b57fc8507ead6-9a9155a47d074f5f9005ddf6337e173e, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678046118198 2023-03-05 19:55:44,141 DEBUG [RS:0;jenkins-hbase4:37037-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=25af79a83c9b248c956b57fc8507ead6-24f57a619907417b8e8a0222c800bc07, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678046119975 2023-03-05 19:55:44,141 DEBUG [RS:0;jenkins-hbase4:37037-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=25af79a83c9b248c956b57fc8507ead6-3004199e82364ee9abaf18f0f843fc76, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678046121774 2023-03-05 19:55:44,141 DEBUG [RS:0;jenkins-hbase4:37037-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=25af79a83c9b248c956b57fc8507ead6-6b8608e70a864d6e8a4e55c85261adfe, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678046123595 2023-03-05 19:55:44,142 DEBUG [RS:0;jenkins-hbase4:37037-longCompactions-0] compactions.Compactor(207): Compacting 2a347452bcd24b71a1bd7bdd97a27786_SeqId_34_, keycount=16, bloomtype=ROW, size=7.3 K, encoding=NONE, compression=NONE, seqNum=35, earliestPutTs=1678046118198 2023-03-05 19:55:44,152 INFO [RS:0;jenkins-hbase4:37037-longCompactions-0] throttle.PressureAwareThroughputController(145): 25af79a83c9b248c956b57fc8507ead6#family1#compaction#116 average throughput is 1.17 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-05 19:55:44,180 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family2/testCompactionWithHFileLink=daa57ebba7c140c319a2f6facc725e4a-c42309b45f3243f2a021144945ad30f4, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family2/testCompactionWithHFileLink=daa57ebba7c140c319a2f6facc725e4a-c2d4bdd0af534bf8b270644df2f98eaf, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family2/testCompactionWithHFileLink=daa57ebba7c140c319a2f6facc725e4a-13ca0708b05643899307a03f0fcc261a, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family2/testCompactionWithHFileLink=daa57ebba7c140c319a2f6facc725e4a-9173733bfa77411fa7038a412b2f97e1] to archive 2023-03-05 19:55:44,181 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-05 19:55:44,182 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family2/testCompactionWithHFileLink=daa57ebba7c140c319a2f6facc725e4a-c42309b45f3243f2a021144945ad30f4 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family2/testCompactionWithHFileLink=daa57ebba7c140c319a2f6facc725e4a-c42309b45f3243f2a021144945ad30f4 2023-03-05 19:55:44,182 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family2/testCompactionWithHFileLink=daa57ebba7c140c319a2f6facc725e4a-c2d4bdd0af534bf8b270644df2f98eaf to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family2/testCompactionWithHFileLink=daa57ebba7c140c319a2f6facc725e4a-c2d4bdd0af534bf8b270644df2f98eaf 2023-03-05 19:55:44,183 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family2/testCompactionWithHFileLink=daa57ebba7c140c319a2f6facc725e4a-13ca0708b05643899307a03f0fcc261a to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family2/testCompactionWithHFileLink=daa57ebba7c140c319a2f6facc725e4a-13ca0708b05643899307a03f0fcc261a 2023-03-05 19:55:44,184 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family2/testCompactionWithHFileLink=daa57ebba7c140c319a2f6facc725e4a-9173733bfa77411fa7038a412b2f97e1 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/family2/testCompactionWithHFileLink=daa57ebba7c140c319a2f6facc725e4a-9173733bfa77411fa7038a412b2f97e1 2023-03-05 19:55:44,231 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1227): Checking to see if procedure is done pid=104 2023-03-05 19:55:44,233 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'ns:testMinorCompaction', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-03-05 19:55:44,233 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure2.ProcedureExecutor(1029): Stored pid=105, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=ns:testMinorCompaction 2023-03-05 19:55:44,235 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_PRE_OPERATION 2023-03-05 19:55:44,235 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "ns" qualifier: "testMinorCompaction" procId is: 105 2023-03-05 19:55:44,235 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-05 19:55:44,235 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-03-05 19:55:44,237 DEBUG [HFileArchiver-16] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/.tmp/data/ns/testMinorCompaction/6fdfc14e8c700cdbd0a252655cbff4b4 2023-03-05 19:55:44,237 DEBUG [HFileArchiver-17] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/.tmp/data/ns/testMinorCompaction/f5738c13f081c7a8286ce16de3ec8a21 2023-03-05 19:55:44,237 DEBUG [HFileArchiver-18] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/.tmp/data/ns/testMinorCompaction/25ed4806e34b655e8718967079efb560 2023-03-05 19:55:44,237 DEBUG [HFileArchiver-16] backup.HFileArchiver(153): Directory hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/.tmp/data/ns/testMinorCompaction/6fdfc14e8c700cdbd0a252655cbff4b4 empty. 2023-03-05 19:55:44,237 DEBUG [HFileArchiver-17] backup.HFileArchiver(153): Directory hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/.tmp/data/ns/testMinorCompaction/f5738c13f081c7a8286ce16de3ec8a21 empty. 2023-03-05 19:55:44,237 DEBUG [HFileArchiver-18] backup.HFileArchiver(153): Directory hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/.tmp/data/ns/testMinorCompaction/25ed4806e34b655e8718967079efb560 empty. 2023-03-05 19:55:44,238 DEBUG [HFileArchiver-16] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/.tmp/data/ns/testMinorCompaction/6fdfc14e8c700cdbd0a252655cbff4b4 2023-03-05 19:55:44,238 DEBUG [HFileArchiver-17] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/.tmp/data/ns/testMinorCompaction/f5738c13f081c7a8286ce16de3ec8a21 2023-03-05 19:55:44,238 DEBUG [HFileArchiver-18] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/.tmp/data/ns/testMinorCompaction/25ed4806e34b655e8718967079efb560 2023-03-05 19:55:44,238 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived ns:testMinorCompaction regions 2023-03-05 19:55:44,336 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-03-05 19:55:44,537 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-03-05 19:55:44,604 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/.tmp/family1/c39a7c87570a4fdfa506f49cb0dd1ccc as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family1/c39a7c87570a4fdfa506f49cb0dd1ccc 2023-03-05 19:55:44,608 INFO [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 942afd2d0a4ceef025fd5a607956d53d/family1 of 942afd2d0a4ceef025fd5a607956d53d into c39a7c87570a4fdfa506f49cb0dd1ccc(size=7.6 K), total size for store is 14.9 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-05 19:55:44,608 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 942afd2d0a4ceef025fd5a607956d53d: 2023-03-05 19:55:44,608 INFO [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,1,1678046116082.942afd2d0a4ceef025fd5a607956d53d., storeName=942afd2d0a4ceef025fd5a607956d53d/family1, priority=12, startTime=1678046143349; duration=1sec 2023-03-05 19:55:44,608 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=3), splitQueue=0 2023-03-05 19:55:44,609 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 5 compacting, 0 eligible, 16 blocking 2023-03-05 19:55:44,609 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 0 files of size 0 starting at candidate #-1 after considering 0 permutations with 0 in ratio 2023-03-05 19:55:44,609 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.SortedCompactionPolicy(232): Not compacting files because we only have 0 files ready for compaction. Need 3 to initiate. 2023-03-05 19:55:44,609 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.CompactSplit(415): Not compacting testCompactionWithHFileLink,2,1678046116082.25af79a83c9b248c956b57fc8507ead6. because compaction request was cancelled 2023-03-05 19:55:44,609 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-03-05 19:55:44,610 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 20920 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-03-05 19:55:44,611 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.HStore(1912): 25af79a83c9b248c956b57fc8507ead6/family2 is initiating minor compaction (all files) 2023-03-05 19:55:44,611 INFO [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 25af79a83c9b248c956b57fc8507ead6/family2 in testCompactionWithHFileLink,2,1678046116082.25af79a83c9b248c956b57fc8507ead6. 2023-03-05 19:55:44,611 INFO [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family2/testCompactionWithHFileLink=25af79a83c9b248c956b57fc8507ead6-22cbfc0fa2af442f976458cd740e398d, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family2/testCompactionWithHFileLink=25af79a83c9b248c956b57fc8507ead6-e2b047fb6d65462d939ae96ac7375619, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family2/testCompactionWithHFileLink=25af79a83c9b248c956b57fc8507ead6-1dfce094a6164169b84ae1182f0260e1, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family2/testCompactionWithHFileLink=25af79a83c9b248c956b57fc8507ead6-11fd411913d44c5187ed3f273ed56bdd] into tmpdir=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/.tmp, totalSize=20.4 K 2023-03-05 19:55:44,611 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=25af79a83c9b248c956b57fc8507ead6-22cbfc0fa2af442f976458cd740e398d, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678046118198 2023-03-05 19:55:44,612 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=25af79a83c9b248c956b57fc8507ead6-e2b047fb6d65462d939ae96ac7375619, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678046119975 2023-03-05 19:55:44,612 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=25af79a83c9b248c956b57fc8507ead6-1dfce094a6164169b84ae1182f0260e1, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678046121774 2023-03-05 19:55:44,613 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=25af79a83c9b248c956b57fc8507ead6-11fd411913d44c5187ed3f273ed56bdd, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678046123595 2023-03-05 19:55:44,623 INFO [RS:0;jenkins-hbase4:37037-shortCompactions-0] throttle.PressureAwareThroughputController(145): 25af79a83c9b248c956b57fc8507ead6#family2#compaction#117 average throughput is 0.58 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-05 19:55:44,645 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/.tmp/data/ns/testMinorCompaction/.tabledesc/.tableinfo.0000000001 2023-03-05 19:55:44,646 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(7675): creating {ENCODED => 25ed4806e34b655e8718967079efb560, NAME => 'ns:testMinorCompaction,2,1678046144232.25ed4806e34b655e8718967079efb560.', STARTKEY => '2', ENDKEY => ''}, tableDescriptor='ns:testMinorCompaction', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/.tmp 2023-03-05 19:55:44,646 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(7675): creating {ENCODED => f5738c13f081c7a8286ce16de3ec8a21, NAME => 'ns:testMinorCompaction,1,1678046144232.f5738c13f081c7a8286ce16de3ec8a21.', STARTKEY => '1', ENDKEY => '2'}, tableDescriptor='ns:testMinorCompaction', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/.tmp 2023-03-05 19:55:44,646 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(7675): creating {ENCODED => 6fdfc14e8c700cdbd0a252655cbff4b4, NAME => 'ns:testMinorCompaction,,1678046144232.6fdfc14e8c700cdbd0a252655cbff4b4.', STARTKEY => '', ENDKEY => '1'}, tableDescriptor='ns:testMinorCompaction', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/.tmp 2023-03-05 19:55:44,680 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family1/testCompactionWithHFileLink=942afd2d0a4ceef025fd5a607956d53d-2ddb935fe9dc46778a4c450ff10d3c44, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family1/testCompactionWithHFileLink=942afd2d0a4ceef025fd5a607956d53d-dc8108fdcc6a4a0a9720b3e2336cb620, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family1/testCompactionWithHFileLink=942afd2d0a4ceef025fd5a607956d53d-c88f1bd396f4409ca38794d504f5c156, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family1/testCompactionWithHFileLink=942afd2d0a4ceef025fd5a607956d53d-8139f54670e14aa3a572f9bdb2477108] to archive 2023-03-05 19:55:44,681 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-05 19:55:44,682 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family1/testCompactionWithHFileLink=942afd2d0a4ceef025fd5a607956d53d-2ddb935fe9dc46778a4c450ff10d3c44 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family1/testCompactionWithHFileLink=942afd2d0a4ceef025fd5a607956d53d-2ddb935fe9dc46778a4c450ff10d3c44 2023-03-05 19:55:44,683 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family1/testCompactionWithHFileLink=942afd2d0a4ceef025fd5a607956d53d-dc8108fdcc6a4a0a9720b3e2336cb620 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family1/testCompactionWithHFileLink=942afd2d0a4ceef025fd5a607956d53d-dc8108fdcc6a4a0a9720b3e2336cb620 2023-03-05 19:55:44,684 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family1/testCompactionWithHFileLink=942afd2d0a4ceef025fd5a607956d53d-c88f1bd396f4409ca38794d504f5c156 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family1/testCompactionWithHFileLink=942afd2d0a4ceef025fd5a607956d53d-c88f1bd396f4409ca38794d504f5c156 2023-03-05 19:55:44,685 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family1/testCompactionWithHFileLink=942afd2d0a4ceef025fd5a607956d53d-8139f54670e14aa3a572f9bdb2477108 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family1/testCompactionWithHFileLink=942afd2d0a4ceef025fd5a607956d53d-8139f54670e14aa3a572f9bdb2477108 2023-03-05 19:55:44,838 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-03-05 19:55:45,055 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(866): Instantiated ns:testMinorCompaction,2,1678046144232.25ed4806e34b655e8718967079efb560.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-05 19:55:45,055 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1604): Closing 25ed4806e34b655e8718967079efb560, disabling compactions & flushes 2023-03-05 19:55:45,055 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1626): Closing region ns:testMinorCompaction,2,1678046144232.25ed4806e34b655e8718967079efb560. 2023-03-05 19:55:45,055 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1647): Waiting without time limit for close lock on ns:testMinorCompaction,2,1678046144232.25ed4806e34b655e8718967079efb560. 2023-03-05 19:55:45,055 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1714): Acquired close lock on ns:testMinorCompaction,2,1678046144232.25ed4806e34b655e8718967079efb560. after waiting 0 ms 2023-03-05 19:55:45,055 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1724): Updates disabled for region ns:testMinorCompaction,2,1678046144232.25ed4806e34b655e8718967079efb560. 2023-03-05 19:55:45,055 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1838): Closed ns:testMinorCompaction,2,1678046144232.25ed4806e34b655e8718967079efb560. 2023-03-05 19:55:45,055 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1558): Region close journal for 25ed4806e34b655e8718967079efb560: 2023-03-05 19:55:45,056 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(866): Instantiated ns:testMinorCompaction,1,1678046144232.f5738c13f081c7a8286ce16de3ec8a21.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-05 19:55:45,056 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1604): Closing f5738c13f081c7a8286ce16de3ec8a21, disabling compactions & flushes 2023-03-05 19:55:45,056 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1626): Closing region ns:testMinorCompaction,1,1678046144232.f5738c13f081c7a8286ce16de3ec8a21. 2023-03-05 19:55:45,056 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1647): Waiting without time limit for close lock on ns:testMinorCompaction,1,1678046144232.f5738c13f081c7a8286ce16de3ec8a21. 2023-03-05 19:55:45,056 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1714): Acquired close lock on ns:testMinorCompaction,1,1678046144232.f5738c13f081c7a8286ce16de3ec8a21. after waiting 0 ms 2023-03-05 19:55:45,056 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1724): Updates disabled for region ns:testMinorCompaction,1,1678046144232.f5738c13f081c7a8286ce16de3ec8a21. 2023-03-05 19:55:45,056 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1838): Closed ns:testMinorCompaction,1,1678046144232.f5738c13f081c7a8286ce16de3ec8a21. 2023-03-05 19:55:45,056 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1558): Region close journal for f5738c13f081c7a8286ce16de3ec8a21: 2023-03-05 19:55:45,057 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(866): Instantiated ns:testMinorCompaction,,1678046144232.6fdfc14e8c700cdbd0a252655cbff4b4.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-05 19:55:45,057 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1604): Closing 6fdfc14e8c700cdbd0a252655cbff4b4, disabling compactions & flushes 2023-03-05 19:55:45,057 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1626): Closing region ns:testMinorCompaction,,1678046144232.6fdfc14e8c700cdbd0a252655cbff4b4. 2023-03-05 19:55:45,057 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on ns:testMinorCompaction,,1678046144232.6fdfc14e8c700cdbd0a252655cbff4b4. 2023-03-05 19:55:45,057 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1714): Acquired close lock on ns:testMinorCompaction,,1678046144232.6fdfc14e8c700cdbd0a252655cbff4b4. after waiting 0 ms 2023-03-05 19:55:45,057 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1724): Updates disabled for region ns:testMinorCompaction,,1678046144232.6fdfc14e8c700cdbd0a252655cbff4b4. 2023-03-05 19:55:45,057 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1838): Closed ns:testMinorCompaction,,1678046144232.6fdfc14e8c700cdbd0a252655cbff4b4. 2023-03-05 19:55:45,057 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1558): Region close journal for 6fdfc14e8c700cdbd0a252655cbff4b4: 2023-03-05 19:55:45,059 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_ADD_TO_META 2023-03-05 19:55:45,059 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"ns:testMinorCompaction,2,1678046144232.25ed4806e34b655e8718967079efb560.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1678046145059"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678046145059"}]},"ts":"1678046145059"} 2023-03-05 19:55:45,059 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"ns:testMinorCompaction,1,1678046144232.f5738c13f081c7a8286ce16de3ec8a21.","families":{"info":[{"qualifier":"regioninfo","vlen":50,"tag":[],"timestamp":"1678046145059"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678046145059"}]},"ts":"1678046145059"} 2023-03-05 19:55:45,060 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"ns:testMinorCompaction,,1678046144232.6fdfc14e8c700cdbd0a252655cbff4b4.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1678046145059"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678046145059"}]},"ts":"1678046145059"} 2023-03-05 19:55:45,061 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-03-05 19:55:45,062 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-05 19:55:45,062 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"ns:testMinorCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678046145062"}]},"ts":"1678046145062"} 2023-03-05 19:55:45,063 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=ns:testMinorCompaction, state=ENABLING in hbase:meta 2023-03-05 19:55:45,066 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=6fdfc14e8c700cdbd0a252655cbff4b4, ASSIGN}, {pid=107, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=f5738c13f081c7a8286ce16de3ec8a21, ASSIGN}, {pid=108, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=25ed4806e34b655e8718967079efb560, ASSIGN}] 2023-03-05 19:55:45,067 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=6fdfc14e8c700cdbd0a252655cbff4b4, ASSIGN 2023-03-05 19:55:45,067 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=108, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=25ed4806e34b655e8718967079efb560, ASSIGN 2023-03-05 19:55:45,067 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=107, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=f5738c13f081c7a8286ce16de3ec8a21, ASSIGN 2023-03-05 19:55:45,068 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=ns:testMinorCompaction, region=6fdfc14e8c700cdbd0a252655cbff4b4, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,37037,1678046018983; forceNewPlan=false, retain=false 2023-03-05 19:55:45,068 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=107, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=ns:testMinorCompaction, region=f5738c13f081c7a8286ce16de3ec8a21, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,37037,1678046018983; forceNewPlan=false, retain=false 2023-03-05 19:55:45,068 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=108, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=ns:testMinorCompaction, region=25ed4806e34b655e8718967079efb560, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,37037,1678046018983; forceNewPlan=false, retain=false 2023-03-05 19:55:45,220 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=108 updating hbase:meta row=25ed4806e34b655e8718967079efb560, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:45,220 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=6fdfc14e8c700cdbd0a252655cbff4b4, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:45,220 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=107 updating hbase:meta row=f5738c13f081c7a8286ce16de3ec8a21, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:45,220 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"ns:testMinorCompaction,2,1678046144232.25ed4806e34b655e8718967079efb560.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1678046145220"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678046145220"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678046145220"}]},"ts":"1678046145220"} 2023-03-05 19:55:45,220 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"ns:testMinorCompaction,1,1678046144232.f5738c13f081c7a8286ce16de3ec8a21.","families":{"info":[{"qualifier":"regioninfo","vlen":50,"tag":[],"timestamp":"1678046145220"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678046145220"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678046145220"}]},"ts":"1678046145220"} 2023-03-05 19:55:45,220 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"ns:testMinorCompaction,,1678046144232.6fdfc14e8c700cdbd0a252655cbff4b4.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1678046145220"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678046145220"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678046145220"}]},"ts":"1678046145220"} 2023-03-05 19:55:45,222 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=109, ppid=108, state=RUNNABLE; OpenRegionProcedure 25ed4806e34b655e8718967079efb560, server=jenkins-hbase4.apache.org,37037,1678046018983}] 2023-03-05 19:55:45,222 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=110, ppid=107, state=RUNNABLE; OpenRegionProcedure f5738c13f081c7a8286ce16de3ec8a21, server=jenkins-hbase4.apache.org,37037,1678046018983}] 2023-03-05 19:55:45,223 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=111, ppid=106, state=RUNNABLE; OpenRegionProcedure 6fdfc14e8c700cdbd0a252655cbff4b4, server=jenkins-hbase4.apache.org,37037,1678046018983}] 2023-03-05 19:55:45,339 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-03-05 19:55:45,368 DEBUG [RS:0;jenkins-hbase4:37037-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/.tmp/family1/fd423513f01c40378175d4f82a8099fe as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family1/fd423513f01c40378175d4f82a8099fe 2023-03-05 19:55:45,371 INFO [RS:0;jenkins-hbase4:37037-longCompactions-0] regionserver.HStore(1652): Completed compaction of 5 (all) file(s) in 25af79a83c9b248c956b57fc8507ead6/family1 of 25af79a83c9b248c956b57fc8507ead6 into fd423513f01c40378175d4f82a8099fe(size=7.6 K), total size for store is 7.6 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-05 19:55:45,372 DEBUG [RS:0;jenkins-hbase4:37037-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 25af79a83c9b248c956b57fc8507ead6: 2023-03-05 19:55:45,372 INFO [RS:0;jenkins-hbase4:37037-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testCompactionWithHFileLink,2,1678046116082.25af79a83c9b248c956b57fc8507ead6., storeName=family1, fileCount=5, fileSize=28.9 K (5.4 K, 5.4 K, 5.4 K, 5.4 K, 7.3 K), priority=2, time=1678046144074; duration=1sec 2023-03-05 19:55:45,372 DEBUG [RS:0;jenkins-hbase4:37037-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-03-05 19:55:45,372 DEBUG [RS:0;jenkins-hbase4:37037-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-03-05 19:55:45,374 DEBUG [RS:0;jenkins-hbase4:37037-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 20920 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-03-05 19:55:45,374 DEBUG [RS:0;jenkins-hbase4:37037-longCompactions-0] regionserver.HStore(1912): 942afd2d0a4ceef025fd5a607956d53d/family2 is initiating minor compaction (all files) 2023-03-05 19:55:45,374 INFO [RS:0;jenkins-hbase4:37037-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 942afd2d0a4ceef025fd5a607956d53d/family2 in testCompactionWithHFileLink,1,1678046116082.942afd2d0a4ceef025fd5a607956d53d. 2023-03-05 19:55:45,374 INFO [RS:0;jenkins-hbase4:37037-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family2/testCompactionWithHFileLink=942afd2d0a4ceef025fd5a607956d53d-9a13755a01ce4e01b7e321e95b43b8cc, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family2/testCompactionWithHFileLink=942afd2d0a4ceef025fd5a607956d53d-af93404d81154179a4c27fce0e335552, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family2/testCompactionWithHFileLink=942afd2d0a4ceef025fd5a607956d53d-2c3de0e4dce5444ca51da5f66eaf06a1, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family2/testCompactionWithHFileLink=942afd2d0a4ceef025fd5a607956d53d-55d239cb7531440088e28c0c9e6143d3] into tmpdir=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/.tmp, totalSize=20.4 K 2023-03-05 19:55:45,374 DEBUG [RS:0;jenkins-hbase4:37037-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=942afd2d0a4ceef025fd5a607956d53d-9a13755a01ce4e01b7e321e95b43b8cc, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678046118197 2023-03-05 19:55:45,375 DEBUG [RS:0;jenkins-hbase4:37037-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=942afd2d0a4ceef025fd5a607956d53d-af93404d81154179a4c27fce0e335552, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678046119974 2023-03-05 19:55:45,376 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open ns:testMinorCompaction,1,1678046144232.f5738c13f081c7a8286ce16de3ec8a21. 2023-03-05 19:55:45,376 DEBUG [RS:0;jenkins-hbase4:37037-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=942afd2d0a4ceef025fd5a607956d53d-2c3de0e4dce5444ca51da5f66eaf06a1, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678046121773 2023-03-05 19:55:45,376 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => f5738c13f081c7a8286ce16de3ec8a21, NAME => 'ns:testMinorCompaction,1,1678046144232.f5738c13f081c7a8286ce16de3ec8a21.', STARTKEY => '1', ENDKEY => '2'} 2023-03-05 19:55:45,376 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompaction f5738c13f081c7a8286ce16de3ec8a21 2023-03-05 19:55:45,376 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated ns:testMinorCompaction,1,1678046144232.f5738c13f081c7a8286ce16de3ec8a21.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-05 19:55:45,376 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for f5738c13f081c7a8286ce16de3ec8a21 2023-03-05 19:55:45,376 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for f5738c13f081c7a8286ce16de3ec8a21 2023-03-05 19:55:45,376 DEBUG [RS:0;jenkins-hbase4:37037-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=942afd2d0a4ceef025fd5a607956d53d-55d239cb7531440088e28c0c9e6143d3, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678046123595 2023-03-05 19:55:45,377 INFO [StoreOpener-f5738c13f081c7a8286ce16de3ec8a21-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region f5738c13f081c7a8286ce16de3ec8a21 2023-03-05 19:55:45,378 DEBUG [StoreOpener-f5738c13f081c7a8286ce16de3ec8a21-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/f5738c13f081c7a8286ce16de3ec8a21/family1 2023-03-05 19:55:45,378 DEBUG [StoreOpener-f5738c13f081c7a8286ce16de3ec8a21-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/f5738c13f081c7a8286ce16de3ec8a21/family1 2023-03-05 19:55:45,379 INFO [StoreOpener-f5738c13f081c7a8286ce16de3ec8a21-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f5738c13f081c7a8286ce16de3ec8a21 columnFamilyName family1 2023-03-05 19:55:45,380 INFO [StoreOpener-f5738c13f081c7a8286ce16de3ec8a21-1] regionserver.HStore(310): Store=f5738c13f081c7a8286ce16de3ec8a21/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-05 19:55:45,380 INFO [StoreOpener-f5738c13f081c7a8286ce16de3ec8a21-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region f5738c13f081c7a8286ce16de3ec8a21 2023-03-05 19:55:45,382 DEBUG [StoreOpener-f5738c13f081c7a8286ce16de3ec8a21-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/f5738c13f081c7a8286ce16de3ec8a21/family2 2023-03-05 19:55:45,382 DEBUG [StoreOpener-f5738c13f081c7a8286ce16de3ec8a21-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/f5738c13f081c7a8286ce16de3ec8a21/family2 2023-03-05 19:55:45,382 INFO [StoreOpener-f5738c13f081c7a8286ce16de3ec8a21-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f5738c13f081c7a8286ce16de3ec8a21 columnFamilyName family2 2023-03-05 19:55:45,383 INFO [StoreOpener-f5738c13f081c7a8286ce16de3ec8a21-1] regionserver.HStore(310): Store=f5738c13f081c7a8286ce16de3ec8a21/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-05 19:55:45,384 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family1/testCompactionWithHFileLink=25af79a83c9b248c956b57fc8507ead6-9a9155a47d074f5f9005ddf6337e173e, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family1/testCompactionWithHFileLink=25af79a83c9b248c956b57fc8507ead6-24f57a619907417b8e8a0222c800bc07, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family1/testCompactionWithHFileLink=25af79a83c9b248c956b57fc8507ead6-3004199e82364ee9abaf18f0f843fc76, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family1/testCompactionWithHFileLink=25af79a83c9b248c956b57fc8507ead6-6b8608e70a864d6e8a4e55c85261adfe, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family1/2a347452bcd24b71a1bd7bdd97a27786_SeqId_34_] to archive 2023-03-05 19:55:45,384 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-05 19:55:45,384 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/f5738c13f081c7a8286ce16de3ec8a21 2023-03-05 19:55:45,384 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/f5738c13f081c7a8286ce16de3ec8a21 2023-03-05 19:55:45,386 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family1/testCompactionWithHFileLink=25af79a83c9b248c956b57fc8507ead6-9a9155a47d074f5f9005ddf6337e173e to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family1/testCompactionWithHFileLink=25af79a83c9b248c956b57fc8507ead6-9a9155a47d074f5f9005ddf6337e173e 2023-03-05 19:55:45,386 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table ns:testMinorCompaction descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-05 19:55:45,387 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family1/testCompactionWithHFileLink=25af79a83c9b248c956b57fc8507ead6-24f57a619907417b8e8a0222c800bc07 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family1/testCompactionWithHFileLink=25af79a83c9b248c956b57fc8507ead6-24f57a619907417b8e8a0222c800bc07 2023-03-05 19:55:45,388 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for f5738c13f081c7a8286ce16de3ec8a21 2023-03-05 19:55:45,389 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family1/testCompactionWithHFileLink=25af79a83c9b248c956b57fc8507ead6-3004199e82364ee9abaf18f0f843fc76 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family1/testCompactionWithHFileLink=25af79a83c9b248c956b57fc8507ead6-3004199e82364ee9abaf18f0f843fc76 2023-03-05 19:55:45,390 INFO [RS:0;jenkins-hbase4:37037-longCompactions-0] throttle.PressureAwareThroughputController(145): 942afd2d0a4ceef025fd5a607956d53d#family2#compaction#118 average throughput is 0.39 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-05 19:55:45,390 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/f5738c13f081c7a8286ce16de3ec8a21/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-05 19:55:45,390 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family1/testCompactionWithHFileLink=25af79a83c9b248c956b57fc8507ead6-6b8608e70a864d6e8a4e55c85261adfe to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family1/testCompactionWithHFileLink=25af79a83c9b248c956b57fc8507ead6-6b8608e70a864d6e8a4e55c85261adfe 2023-03-05 19:55:45,390 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened f5738c13f081c7a8286ce16de3ec8a21; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71664870, jitterRate=0.06788977980613708}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-05 19:55:45,390 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for f5738c13f081c7a8286ce16de3ec8a21: 2023-03-05 19:55:45,391 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for ns:testMinorCompaction,1,1678046144232.f5738c13f081c7a8286ce16de3ec8a21., pid=110, masterSystemTime=1678046145374 2023-03-05 19:55:45,391 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family1/2a347452bcd24b71a1bd7bdd97a27786_SeqId_34_ to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family1/2a347452bcd24b71a1bd7bdd97a27786_SeqId_34_ 2023-03-05 19:55:45,392 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for ns:testMinorCompaction,1,1678046144232.f5738c13f081c7a8286ce16de3ec8a21. 2023-03-05 19:55:45,392 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened ns:testMinorCompaction,1,1678046144232.f5738c13f081c7a8286ce16de3ec8a21. 2023-03-05 19:55:45,392 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open ns:testMinorCompaction,,1678046144232.6fdfc14e8c700cdbd0a252655cbff4b4. 2023-03-05 19:55:45,393 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 6fdfc14e8c700cdbd0a252655cbff4b4, NAME => 'ns:testMinorCompaction,,1678046144232.6fdfc14e8c700cdbd0a252655cbff4b4.', STARTKEY => '', ENDKEY => '1'} 2023-03-05 19:55:45,393 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=107 updating hbase:meta row=f5738c13f081c7a8286ce16de3ec8a21, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:45,393 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompaction 6fdfc14e8c700cdbd0a252655cbff4b4 2023-03-05 19:55:45,393 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"ns:testMinorCompaction,1,1678046144232.f5738c13f081c7a8286ce16de3ec8a21.","families":{"info":[{"qualifier":"regioninfo","vlen":50,"tag":[],"timestamp":"1678046145393"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678046145393"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678046145393"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678046145393"}]},"ts":"1678046145393"} 2023-03-05 19:55:45,393 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated ns:testMinorCompaction,,1678046144232.6fdfc14e8c700cdbd0a252655cbff4b4.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-05 19:55:45,393 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 6fdfc14e8c700cdbd0a252655cbff4b4 2023-03-05 19:55:45,393 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 6fdfc14e8c700cdbd0a252655cbff4b4 2023-03-05 19:55:45,394 INFO [StoreOpener-6fdfc14e8c700cdbd0a252655cbff4b4-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 6fdfc14e8c700cdbd0a252655cbff4b4 2023-03-05 19:55:45,396 DEBUG [StoreOpener-6fdfc14e8c700cdbd0a252655cbff4b4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/6fdfc14e8c700cdbd0a252655cbff4b4/family1 2023-03-05 19:55:45,396 DEBUG [StoreOpener-6fdfc14e8c700cdbd0a252655cbff4b4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/6fdfc14e8c700cdbd0a252655cbff4b4/family1 2023-03-05 19:55:45,396 INFO [StoreOpener-6fdfc14e8c700cdbd0a252655cbff4b4-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6fdfc14e8c700cdbd0a252655cbff4b4 columnFamilyName family1 2023-03-05 19:55:45,396 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=110, resume processing ppid=107 2023-03-05 19:55:45,396 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=110, ppid=107, state=SUCCESS; OpenRegionProcedure f5738c13f081c7a8286ce16de3ec8a21, server=jenkins-hbase4.apache.org,37037,1678046018983 in 172 msec 2023-03-05 19:55:45,397 INFO [StoreOpener-6fdfc14e8c700cdbd0a252655cbff4b4-1] regionserver.HStore(310): Store=6fdfc14e8c700cdbd0a252655cbff4b4/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-05 19:55:45,397 INFO [StoreOpener-6fdfc14e8c700cdbd0a252655cbff4b4-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 6fdfc14e8c700cdbd0a252655cbff4b4 2023-03-05 19:55:45,397 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=107, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=ns:testMinorCompaction, region=f5738c13f081c7a8286ce16de3ec8a21, ASSIGN in 330 msec 2023-03-05 19:55:45,398 DEBUG [StoreOpener-6fdfc14e8c700cdbd0a252655cbff4b4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/6fdfc14e8c700cdbd0a252655cbff4b4/family2 2023-03-05 19:55:45,398 DEBUG [StoreOpener-6fdfc14e8c700cdbd0a252655cbff4b4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/6fdfc14e8c700cdbd0a252655cbff4b4/family2 2023-03-05 19:55:45,399 INFO [StoreOpener-6fdfc14e8c700cdbd0a252655cbff4b4-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6fdfc14e8c700cdbd0a252655cbff4b4 columnFamilyName family2 2023-03-05 19:55:45,399 INFO [StoreOpener-6fdfc14e8c700cdbd0a252655cbff4b4-1] regionserver.HStore(310): Store=6fdfc14e8c700cdbd0a252655cbff4b4/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-05 19:55:45,400 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/6fdfc14e8c700cdbd0a252655cbff4b4 2023-03-05 19:55:45,400 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/6fdfc14e8c700cdbd0a252655cbff4b4 2023-03-05 19:55:45,402 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table ns:testMinorCompaction descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-05 19:55:45,403 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 6fdfc14e8c700cdbd0a252655cbff4b4 2023-03-05 19:55:45,404 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/6fdfc14e8c700cdbd0a252655cbff4b4/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-05 19:55:45,405 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 6fdfc14e8c700cdbd0a252655cbff4b4; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65163898, jitterRate=-0.0289822518825531}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-05 19:55:45,405 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 6fdfc14e8c700cdbd0a252655cbff4b4: 2023-03-05 19:55:45,405 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for ns:testMinorCompaction,,1678046144232.6fdfc14e8c700cdbd0a252655cbff4b4., pid=111, masterSystemTime=1678046145374 2023-03-05 19:55:45,406 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for ns:testMinorCompaction,,1678046144232.6fdfc14e8c700cdbd0a252655cbff4b4. 2023-03-05 19:55:45,406 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened ns:testMinorCompaction,,1678046144232.6fdfc14e8c700cdbd0a252655cbff4b4. 2023-03-05 19:55:45,407 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open ns:testMinorCompaction,2,1678046144232.25ed4806e34b655e8718967079efb560. 2023-03-05 19:55:45,407 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 25ed4806e34b655e8718967079efb560, NAME => 'ns:testMinorCompaction,2,1678046144232.25ed4806e34b655e8718967079efb560.', STARTKEY => '2', ENDKEY => ''} 2023-03-05 19:55:45,407 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=6fdfc14e8c700cdbd0a252655cbff4b4, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:45,407 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"ns:testMinorCompaction,,1678046144232.6fdfc14e8c700cdbd0a252655cbff4b4.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1678046145407"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678046145407"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678046145407"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678046145407"}]},"ts":"1678046145407"} 2023-03-05 19:55:45,407 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompaction 25ed4806e34b655e8718967079efb560 2023-03-05 19:55:45,407 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated ns:testMinorCompaction,2,1678046144232.25ed4806e34b655e8718967079efb560.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-05 19:55:45,407 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 25ed4806e34b655e8718967079efb560 2023-03-05 19:55:45,407 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 25ed4806e34b655e8718967079efb560 2023-03-05 19:55:45,408 INFO [StoreOpener-25ed4806e34b655e8718967079efb560-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 25ed4806e34b655e8718967079efb560 2023-03-05 19:55:45,409 DEBUG [StoreOpener-25ed4806e34b655e8718967079efb560-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/25ed4806e34b655e8718967079efb560/family1 2023-03-05 19:55:45,409 DEBUG [StoreOpener-25ed4806e34b655e8718967079efb560-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/25ed4806e34b655e8718967079efb560/family1 2023-03-05 19:55:45,409 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=111, resume processing ppid=106 2023-03-05 19:55:45,409 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=111, ppid=106, state=SUCCESS; OpenRegionProcedure 6fdfc14e8c700cdbd0a252655cbff4b4, server=jenkins-hbase4.apache.org,37037,1678046018983 in 185 msec 2023-03-05 19:55:45,409 INFO [StoreOpener-25ed4806e34b655e8718967079efb560-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 25ed4806e34b655e8718967079efb560 columnFamilyName family1 2023-03-05 19:55:45,410 INFO [StoreOpener-25ed4806e34b655e8718967079efb560-1] regionserver.HStore(310): Store=25ed4806e34b655e8718967079efb560/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-05 19:55:45,410 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=106, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=ns:testMinorCompaction, region=6fdfc14e8c700cdbd0a252655cbff4b4, ASSIGN in 343 msec 2023-03-05 19:55:45,410 INFO [StoreOpener-25ed4806e34b655e8718967079efb560-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 25ed4806e34b655e8718967079efb560 2023-03-05 19:55:45,411 DEBUG [StoreOpener-25ed4806e34b655e8718967079efb560-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/25ed4806e34b655e8718967079efb560/family2 2023-03-05 19:55:45,411 DEBUG [StoreOpener-25ed4806e34b655e8718967079efb560-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/25ed4806e34b655e8718967079efb560/family2 2023-03-05 19:55:45,412 INFO [StoreOpener-25ed4806e34b655e8718967079efb560-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 25ed4806e34b655e8718967079efb560 columnFamilyName family2 2023-03-05 19:55:45,412 INFO [StoreOpener-25ed4806e34b655e8718967079efb560-1] regionserver.HStore(310): Store=25ed4806e34b655e8718967079efb560/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-05 19:55:45,413 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/25ed4806e34b655e8718967079efb560 2023-03-05 19:55:45,414 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/25ed4806e34b655e8718967079efb560 2023-03-05 19:55:45,415 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table ns:testMinorCompaction descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-05 19:55:45,416 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 25ed4806e34b655e8718967079efb560 2023-03-05 19:55:45,417 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/25ed4806e34b655e8718967079efb560/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-05 19:55:45,417 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 25ed4806e34b655e8718967079efb560; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71382932, jitterRate=0.06368857622146606}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-05 19:55:45,417 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 25ed4806e34b655e8718967079efb560: 2023-03-05 19:55:45,418 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for ns:testMinorCompaction,2,1678046144232.25ed4806e34b655e8718967079efb560., pid=109, masterSystemTime=1678046145374 2023-03-05 19:55:45,419 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for ns:testMinorCompaction,2,1678046144232.25ed4806e34b655e8718967079efb560. 2023-03-05 19:55:45,419 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened ns:testMinorCompaction,2,1678046144232.25ed4806e34b655e8718967079efb560. 2023-03-05 19:55:45,419 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=108 updating hbase:meta row=25ed4806e34b655e8718967079efb560, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:45,419 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"ns:testMinorCompaction,2,1678046144232.25ed4806e34b655e8718967079efb560.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1678046145419"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678046145419"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678046145419"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678046145419"}]},"ts":"1678046145419"} 2023-03-05 19:55:45,421 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=109, resume processing ppid=108 2023-03-05 19:55:45,421 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=109, ppid=108, state=SUCCESS; OpenRegionProcedure 25ed4806e34b655e8718967079efb560, server=jenkins-hbase4.apache.org,37037,1678046018983 in 198 msec 2023-03-05 19:55:45,422 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=108, resume processing ppid=105 2023-03-05 19:55:45,422 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=108, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=ns:testMinorCompaction, region=25ed4806e34b655e8718967079efb560, ASSIGN in 355 msec 2023-03-05 19:55:45,422 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-05 19:55:45,423 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"ns:testMinorCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678046145422"}]},"ts":"1678046145422"} 2023-03-05 19:55:45,423 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=ns:testMinorCompaction, state=ENABLED in hbase:meta 2023-03-05 19:55:45,426 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_POST_OPERATION 2023-03-05 19:55:45,427 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=105, state=SUCCESS; CreateTableProcedure table=ns:testMinorCompaction in 1.1930 sec 2023-03-05 19:55:45,483 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-05 19:55:45,839 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/.tmp/family2/a9bb7919e37f43799d885a2f61a7984c as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family2/a9bb7919e37f43799d885a2f61a7984c 2023-03-05 19:55:45,843 INFO [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 25af79a83c9b248c956b57fc8507ead6/family2 of 25af79a83c9b248c956b57fc8507ead6 into a9bb7919e37f43799d885a2f61a7984c(size=6.4 K), total size for store is 6.4 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-05 19:55:45,843 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 25af79a83c9b248c956b57fc8507ead6: 2023-03-05 19:55:45,843 INFO [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,2,1678046116082.25af79a83c9b248c956b57fc8507ead6., storeName=25af79a83c9b248c956b57fc8507ead6/family2, priority=12, startTime=1678046143411; duration=1sec 2023-03-05 19:55:45,843 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-05 19:55:45,880 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family2/testCompactionWithHFileLink=25af79a83c9b248c956b57fc8507ead6-22cbfc0fa2af442f976458cd740e398d, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family2/testCompactionWithHFileLink=25af79a83c9b248c956b57fc8507ead6-e2b047fb6d65462d939ae96ac7375619, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family2/testCompactionWithHFileLink=25af79a83c9b248c956b57fc8507ead6-1dfce094a6164169b84ae1182f0260e1, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family2/testCompactionWithHFileLink=25af79a83c9b248c956b57fc8507ead6-11fd411913d44c5187ed3f273ed56bdd] to archive 2023-03-05 19:55:45,881 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-05 19:55:45,882 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family2/testCompactionWithHFileLink=25af79a83c9b248c956b57fc8507ead6-22cbfc0fa2af442f976458cd740e398d to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family2/testCompactionWithHFileLink=25af79a83c9b248c956b57fc8507ead6-22cbfc0fa2af442f976458cd740e398d 2023-03-05 19:55:45,883 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family2/testCompactionWithHFileLink=25af79a83c9b248c956b57fc8507ead6-e2b047fb6d65462d939ae96ac7375619 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family2/testCompactionWithHFileLink=25af79a83c9b248c956b57fc8507ead6-e2b047fb6d65462d939ae96ac7375619 2023-03-05 19:55:45,884 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family2/testCompactionWithHFileLink=25af79a83c9b248c956b57fc8507ead6-1dfce094a6164169b84ae1182f0260e1 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family2/testCompactionWithHFileLink=25af79a83c9b248c956b57fc8507ead6-1dfce094a6164169b84ae1182f0260e1 2023-03-05 19:55:45,885 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family2/testCompactionWithHFileLink=25af79a83c9b248c956b57fc8507ead6-11fd411913d44c5187ed3f273ed56bdd to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/family2/testCompactionWithHFileLink=25af79a83c9b248c956b57fc8507ead6-11fd411913d44c5187ed3f273ed56bdd 2023-03-05 19:55:46,207 DEBUG [RS:0;jenkins-hbase4:37037-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/.tmp/family2/ec99534ad4224ad9a0c459af11b2a9ee as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family2/ec99534ad4224ad9a0c459af11b2a9ee 2023-03-05 19:55:46,211 INFO [RS:0;jenkins-hbase4:37037-longCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 942afd2d0a4ceef025fd5a607956d53d/family2 of 942afd2d0a4ceef025fd5a607956d53d into ec99534ad4224ad9a0c459af11b2a9ee(size=6.4 K), total size for store is 6.4 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-05 19:55:46,211 DEBUG [RS:0;jenkins-hbase4:37037-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 942afd2d0a4ceef025fd5a607956d53d: 2023-03-05 19:55:46,211 INFO [RS:0;jenkins-hbase4:37037-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,1,1678046116082.942afd2d0a4ceef025fd5a607956d53d., storeName=942afd2d0a4ceef025fd5a607956d53d/family2, priority=12, startTime=1678046143350; duration=0sec 2023-03-05 19:55:46,211 DEBUG [RS:0;jenkins-hbase4:37037-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-05 19:55:46,283 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family2/testCompactionWithHFileLink=942afd2d0a4ceef025fd5a607956d53d-9a13755a01ce4e01b7e321e95b43b8cc, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family2/testCompactionWithHFileLink=942afd2d0a4ceef025fd5a607956d53d-af93404d81154179a4c27fce0e335552, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family2/testCompactionWithHFileLink=942afd2d0a4ceef025fd5a607956d53d-2c3de0e4dce5444ca51da5f66eaf06a1, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family2/testCompactionWithHFileLink=942afd2d0a4ceef025fd5a607956d53d-55d239cb7531440088e28c0c9e6143d3] to archive 2023-03-05 19:55:46,284 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-05 19:55:46,286 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family2/testCompactionWithHFileLink=942afd2d0a4ceef025fd5a607956d53d-9a13755a01ce4e01b7e321e95b43b8cc to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family2/testCompactionWithHFileLink=942afd2d0a4ceef025fd5a607956d53d-9a13755a01ce4e01b7e321e95b43b8cc 2023-03-05 19:55:46,287 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family2/testCompactionWithHFileLink=942afd2d0a4ceef025fd5a607956d53d-af93404d81154179a4c27fce0e335552 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family2/testCompactionWithHFileLink=942afd2d0a4ceef025fd5a607956d53d-af93404d81154179a4c27fce0e335552 2023-03-05 19:55:46,288 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family2/testCompactionWithHFileLink=942afd2d0a4ceef025fd5a607956d53d-2c3de0e4dce5444ca51da5f66eaf06a1 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family2/testCompactionWithHFileLink=942afd2d0a4ceef025fd5a607956d53d-2c3de0e4dce5444ca51da5f66eaf06a1 2023-03-05 19:55:46,289 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family2/testCompactionWithHFileLink=942afd2d0a4ceef025fd5a607956d53d-55d239cb7531440088e28c0c9e6143d3 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/family2/testCompactionWithHFileLink=942afd2d0a4ceef025fd5a607956d53d-55d239cb7531440088e28c0c9e6143d3 2023-03-05 19:55:46,340 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-03-05 19:55:46,340 INFO [Listener at localhost/40787] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: ns:testMinorCompaction, procId: 105 completed 2023-03-05 19:55:46,346 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,,1678046144232.6fdfc14e8c700cdbd0a252655cbff4b4. with WAL disabled. Data may be lost in the event of a crash. 2023-03-05 19:55:46,346 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,1,1678046144232.f5738c13f081c7a8286ce16de3ec8a21. with WAL disabled. Data may be lost in the event of a crash. 2023-03-05 19:55:46,347 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,2,1678046144232.25ed4806e34b655e8718967079efb560. with WAL disabled. Data may be lost in the event of a crash. 2023-03-05 19:55:46,348 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-05 19:55:46,350 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-03-05 19:55:46,350 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-03-05 19:55:46,350 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:55:46,350 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-03-05 19:55:46,350 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-03-05 19:55:46,350 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-05 19:55:46,350 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-05 19:55:46,352 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:55:46,352 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:46,352 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:55:46,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:55:46,352 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:46,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-05 19:55:46,352 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-05 19:55:46,352 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-05 19:55:46,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-05 19:55:46,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-05 19:55:46,353 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-03-05 19:55:46,353 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-03-05 19:55:46,353 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-03-05 19:55:46,353 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:55:46,353 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-03-05 19:55:46,354 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1678046144232.25ed4806e34b655e8718967079efb560. 2023-03-05 19:55:46,354 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1678046144232.25ed4806e34b655e8718967079efb560. started... 2023-03-05 19:55:46,354 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-05 19:55:46,354 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-05 19:55:46,354 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1678046144232.f5738c13f081c7a8286ce16de3ec8a21. 2023-03-05 19:55:46,355 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 25ed4806e34b655e8718967079efb560 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-03-05 19:55:46,356 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1678046144232.f5738c13f081c7a8286ce16de3ec8a21. started... 2023-03-05 19:55:46,354 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1678046144232.6fdfc14e8c700cdbd0a252655cbff4b4. 2023-03-05 19:55:46,356 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1678046144232.6fdfc14e8c700cdbd0a252655cbff4b4. started... 2023-03-05 19:55:46,356 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing f5738c13f081c7a8286ce16de3ec8a21 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-03-05 19:55:46,356 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 6fdfc14e8c700cdbd0a252655cbff4b4 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-03-05 19:55:46,367 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230305b514ad0f1fb24aba8224f018265df055 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c20230305b514ad0f1fb24aba8224f018265df055 2023-03-05 19:55:46,767 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303057ca0feb1e1784191832759b3dc4ad143 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e202303057ca0feb1e1784191832759b3dc4ad143 2023-03-05 19:55:46,767 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303055525fb319dd04a3ba26b40c244794c22 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b202303055525fb319dd04a3ba26b40c244794c22 2023-03-05 19:55:46,771 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/25ed4806e34b655e8718967079efb560/.tmp/family1/1c5ca09a26864f08a5625671975ef036 2023-03-05 19:55:46,777 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/f5738c13f081c7a8286ce16de3ec8a21/.tmp/family1/7ea37ba56aeb4d21a21a5d3bf889223c 2023-03-05 19:55:46,787 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230305eced43122613404690ec0c36fa5f3f72 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c20230305eced43122613404690ec0c36fa5f3f72 2023-03-05 19:55:47,172 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/6fdfc14e8c700cdbd0a252655cbff4b4/.tmp/family1/5032d1ccb70d49d6a55d3e09c1b22529 2023-03-05 19:55:47,177 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'ns:testMinorCompaction' 2023-03-05 19:55:47,198 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/25ed4806e34b655e8718967079efb560/.tmp/family2/a4dcc57fdaee455db32e516ed78bb0bc 2023-03-05 19:55:47,201 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230305f92ae5efd5354852bce892e180dc52ec to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b20230305f92ae5efd5354852bce892e180dc52ec 2023-03-05 19:55:47,202 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/25ed4806e34b655e8718967079efb560/.tmp/family1/1c5ca09a26864f08a5625671975ef036 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/25ed4806e34b655e8718967079efb560/family1/1c5ca09a26864f08a5625671975ef036 2023-03-05 19:55:47,205 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/25ed4806e34b655e8718967079efb560/family1/1c5ca09a26864f08a5625671975ef036, entries=4, sequenceid=5, filesize=5.4 K 2023-03-05 19:55:47,206 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/25ed4806e34b655e8718967079efb560/.tmp/family2/a4dcc57fdaee455db32e516ed78bb0bc as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/25ed4806e34b655e8718967079efb560/family2/a4dcc57fdaee455db32e516ed78bb0bc 2023-03-05 19:55:47,209 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/25ed4806e34b655e8718967079efb560/family2/a4dcc57fdaee455db32e516ed78bb0bc, entries=2, sequenceid=5, filesize=5.1 K 2023-03-05 19:55:47,209 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 25ed4806e34b655e8718967079efb560 in 854ms, sequenceid=5, compaction requested=false 2023-03-05 19:55:47,210 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 25ed4806e34b655e8718967079efb560: 2023-03-05 19:55:47,210 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1678046144232.25ed4806e34b655e8718967079efb560. 2023-03-05 19:55:47,210 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-05 19:55:47,586 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303058350ad497d2848e390d4b2cce8390732 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e202303058350ad497d2848e390d4b2cce8390732 2023-03-05 19:55:47,606 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/f5738c13f081c7a8286ce16de3ec8a21/.tmp/family2/da7d211a0d2146d98ca748102a227893 2023-03-05 19:55:47,609 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/f5738c13f081c7a8286ce16de3ec8a21/.tmp/family1/7ea37ba56aeb4d21a21a5d3bf889223c as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/f5738c13f081c7a8286ce16de3ec8a21/family1/7ea37ba56aeb4d21a21a5d3bf889223c 2023-03-05 19:55:47,613 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/f5738c13f081c7a8286ce16de3ec8a21/family1/7ea37ba56aeb4d21a21a5d3bf889223c, entries=4, sequenceid=5, filesize=5.4 K 2023-03-05 19:55:47,613 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/f5738c13f081c7a8286ce16de3ec8a21/.tmp/family2/da7d211a0d2146d98ca748102a227893 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/f5738c13f081c7a8286ce16de3ec8a21/family2/da7d211a0d2146d98ca748102a227893 2023-03-05 19:55:47,617 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/f5738c13f081c7a8286ce16de3ec8a21/family2/da7d211a0d2146d98ca748102a227893, entries=2, sequenceid=5, filesize=5.1 K 2023-03-05 19:55:47,617 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for f5738c13f081c7a8286ce16de3ec8a21 in 1261ms, sequenceid=5, compaction requested=false 2023-03-05 19:55:47,618 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for f5738c13f081c7a8286ce16de3ec8a21: 2023-03-05 19:55:47,618 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1678046144232.f5738c13f081c7a8286ce16de3ec8a21. 2023-03-05 19:55:47,618 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-05 19:55:47,991 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/6fdfc14e8c700cdbd0a252655cbff4b4/.tmp/family2/7bedf8a039ea4580bbf505c8d84bd02f 2023-03-05 19:55:47,995 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/6fdfc14e8c700cdbd0a252655cbff4b4/.tmp/family1/5032d1ccb70d49d6a55d3e09c1b22529 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/6fdfc14e8c700cdbd0a252655cbff4b4/family1/5032d1ccb70d49d6a55d3e09c1b22529 2023-03-05 19:55:47,998 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/6fdfc14e8c700cdbd0a252655cbff4b4/family1/5032d1ccb70d49d6a55d3e09c1b22529, entries=4, sequenceid=5, filesize=5.4 K 2023-03-05 19:55:47,998 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/6fdfc14e8c700cdbd0a252655cbff4b4/.tmp/family2/7bedf8a039ea4580bbf505c8d84bd02f as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/6fdfc14e8c700cdbd0a252655cbff4b4/family2/7bedf8a039ea4580bbf505c8d84bd02f 2023-03-05 19:55:48,001 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/6fdfc14e8c700cdbd0a252655cbff4b4/family2/7bedf8a039ea4580bbf505c8d84bd02f, entries=2, sequenceid=5, filesize=5.1 K 2023-03-05 19:55:48,002 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 6fdfc14e8c700cdbd0a252655cbff4b4 in 1646ms, sequenceid=5, compaction requested=false 2023-03-05 19:55:48,002 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 6fdfc14e8c700cdbd0a252655cbff4b4: 2023-03-05 19:55:48,002 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1678046144232.6fdfc14e8c700cdbd0a252655cbff4b4. 2023-03-05 19:55:48,002 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-05 19:55:48,002 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-05 19:55:48,002 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:48,002 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-03-05 19:55:48,002 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-03-05 19:55:48,004 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:48,004 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-05 19:55:48,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:48,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:55:48,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:55:48,004 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-05 19:55:48,004 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-05 19:55:48,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:55:48,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:55:48,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-05 19:55:48,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:48,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:55:48,005 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-03-05 19:55:48,005 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3b4345da[Count = 0] remaining members to acquire global barrier 2023-03-05 19:55:48,005 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-03-05 19:55:48,005 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-05 19:55:48,006 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-05 19:55:48,007 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-05 19:55:48,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-05 19:55:48,007 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-03-05 19:55:48,007 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-03-05 19:55:48,007 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,37037,1678046018983' in zk 2023-03-05 19:55:48,007 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:48,007 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-05 19:55:48,009 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:48,009 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-03-05 19:55:48,009 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:55:48,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:48,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:55:48,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:55:48,009 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-03-05 19:55:48,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:55:48,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:55:48,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-05 19:55:48,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:48,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:55:48,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-05 19:55:48,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:48,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase4.apache.org,37037,1678046018983': 2023-03-05 19:55:48,011 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-03-05 19:55:48,011 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-03-05 19:55:48,011 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-05 19:55:48,011 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-05 19:55:48,011 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-03-05 19:55:48,011 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-05 19:55:48,012 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-05 19:55:48,012 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-05 19:55:48,012 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-05 19:55:48,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-05 19:55:48,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:55:48,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:55:48,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-05 19:55:48,012 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:55:48,013 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:55:48,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:55:48,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:55:48,013 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:48,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-05 19:55:48,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:55:48,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-05 19:55:48,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-05 19:55:48,014 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:48,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:48,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:55:48,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-05 19:55:48,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:48,016 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:48,016 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:55:48,016 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-05 19:55:48,016 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:55:48,016 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-03-05 19:55:48,016 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-05 19:55:48,016 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-05 19:55:48,016 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:48,016 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-05 19:55:48,017 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-05 19:55:48,017 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-05 19:55:48,016 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:55:48,016 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:55:48,016 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/ns:testMinorCompaction because node does not exist (not an error) 2023-03-05 19:55:48,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:55:48,017 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-03-05 19:55:48,017 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-05 19:55:48,017 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:55:48,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:55:48,117 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-05 19:55:48,118 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-05 19:55:48,119 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,,1678046144232.6fdfc14e8c700cdbd0a252655cbff4b4. with WAL disabled. Data may be lost in the event of a crash. 2023-03-05 19:55:48,119 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,1,1678046144232.f5738c13f081c7a8286ce16de3ec8a21. with WAL disabled. Data may be lost in the event of a crash. 2023-03-05 19:55:48,119 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,2,1678046144232.25ed4806e34b655e8718967079efb560. with WAL disabled. Data may be lost in the event of a crash. 2023-03-05 19:55:48,121 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-05 19:55:48,122 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(143): Procedure ns:testMinorCompaction was in running list but was completed. Accepting new attempt. 2023-03-05 19:55:48,123 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-03-05 19:55:48,123 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-03-05 19:55:48,123 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:55:48,123 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-03-05 19:55:48,123 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-03-05 19:55:48,123 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-05 19:55:48,123 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-05 19:55:48,124 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:55:48,124 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:48,124 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:55:48,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:55:48,125 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:48,125 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-05 19:55:48,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-05 19:55:48,125 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-05 19:55:48,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-05 19:55:48,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-05 19:55:48,125 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-03-05 19:55:48,125 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc ns:testMinorCompaction is still present, removing 2023-03-05 19:55:48,125 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-03-05 19:55:48,125 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-03-05 19:55:48,125 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:55:48,125 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-03-05 19:55:48,126 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1678046144232.25ed4806e34b655e8718967079efb560. 2023-03-05 19:55:48,126 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-05 19:55:48,126 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-05 19:55:48,126 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1678046144232.f5738c13f081c7a8286ce16de3ec8a21. 2023-03-05 19:55:48,126 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1678046144232.f5738c13f081c7a8286ce16de3ec8a21. started... 2023-03-05 19:55:48,126 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1678046144232.25ed4806e34b655e8718967079efb560. started... 2023-03-05 19:55:48,126 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1678046144232.6fdfc14e8c700cdbd0a252655cbff4b4. 2023-03-05 19:55:48,126 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1678046144232.6fdfc14e8c700cdbd0a252655cbff4b4. started... 2023-03-05 19:55:48,126 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing f5738c13f081c7a8286ce16de3ec8a21 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-03-05 19:55:48,126 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 25ed4806e34b655e8718967079efb560 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-03-05 19:55:48,126 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 6fdfc14e8c700cdbd0a252655cbff4b4 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-03-05 19:55:48,536 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230305cc7b720ff6d34e9286cab923ed15fad8 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b20230305cc7b720ff6d34e9286cab923ed15fad8 2023-03-05 19:55:48,541 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230305d93b21ae85f847c397cf8a9395b46def to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e20230305d93b21ae85f847c397cf8a9395b46def 2023-03-05 19:55:48,541 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230305689bdbe5b9ee4b1c9380f21301ec84d5 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c20230305689bdbe5b9ee4b1c9380f21301ec84d5 2023-03-05 19:55:48,942 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/f5738c13f081c7a8286ce16de3ec8a21/.tmp/family1/5eeef2442b714333896ff4ee143861ae 2023-03-05 19:55:48,945 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/6fdfc14e8c700cdbd0a252655cbff4b4/.tmp/family1/099d290778644902a97fce25291e5726 2023-03-05 19:55:48,946 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/25ed4806e34b655e8718967079efb560/.tmp/family1/959ce2e3e340495cbf38b70641ea0a02 2023-03-05 19:55:48,959 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303053754da853de245188b8690ee5cf73cf9 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e202303053754da853de245188b8690ee5cf73cf9 2023-03-05 19:55:48,959 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230305991129dfaaef426a9468556950114d86 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c20230305991129dfaaef426a9468556950114d86 2023-03-05 19:55:49,355 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230305983b750e79404989a13b926d4ff8622c to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b20230305983b750e79404989a13b926d4ff8622c 2023-03-05 19:55:49,368 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/25ed4806e34b655e8718967079efb560/.tmp/family2/7168e07305da425b895922ef287d801c 2023-03-05 19:55:49,368 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/6fdfc14e8c700cdbd0a252655cbff4b4/.tmp/family2/df0f37123f2d4c53988bfc6eef9a8b92 2023-03-05 19:55:49,371 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/25ed4806e34b655e8718967079efb560/.tmp/family1/959ce2e3e340495cbf38b70641ea0a02 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/25ed4806e34b655e8718967079efb560/family1/959ce2e3e340495cbf38b70641ea0a02 2023-03-05 19:55:49,371 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/6fdfc14e8c700cdbd0a252655cbff4b4/.tmp/family1/099d290778644902a97fce25291e5726 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/6fdfc14e8c700cdbd0a252655cbff4b4/family1/099d290778644902a97fce25291e5726 2023-03-05 19:55:49,375 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/6fdfc14e8c700cdbd0a252655cbff4b4/family1/099d290778644902a97fce25291e5726, entries=4, sequenceid=9, filesize=5.4 K 2023-03-05 19:55:49,375 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/25ed4806e34b655e8718967079efb560/family1/959ce2e3e340495cbf38b70641ea0a02, entries=4, sequenceid=9, filesize=5.4 K 2023-03-05 19:55:49,376 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/6fdfc14e8c700cdbd0a252655cbff4b4/.tmp/family2/df0f37123f2d4c53988bfc6eef9a8b92 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/6fdfc14e8c700cdbd0a252655cbff4b4/family2/df0f37123f2d4c53988bfc6eef9a8b92 2023-03-05 19:55:49,376 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/25ed4806e34b655e8718967079efb560/.tmp/family2/7168e07305da425b895922ef287d801c as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/25ed4806e34b655e8718967079efb560/family2/7168e07305da425b895922ef287d801c 2023-03-05 19:55:49,379 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/25ed4806e34b655e8718967079efb560/family2/7168e07305da425b895922ef287d801c, entries=2, sequenceid=9, filesize=5.1 K 2023-03-05 19:55:49,379 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/6fdfc14e8c700cdbd0a252655cbff4b4/family2/df0f37123f2d4c53988bfc6eef9a8b92, entries=2, sequenceid=9, filesize=5.1 K 2023-03-05 19:55:49,380 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 25ed4806e34b655e8718967079efb560 in 1254ms, sequenceid=9, compaction requested=false 2023-03-05 19:55:49,380 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 25ed4806e34b655e8718967079efb560: 2023-03-05 19:55:49,380 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1678046144232.25ed4806e34b655e8718967079efb560. 2023-03-05 19:55:49,380 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-05 19:55:49,380 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 6fdfc14e8c700cdbd0a252655cbff4b4 in 1254ms, sequenceid=9, compaction requested=false 2023-03-05 19:55:49,380 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 6fdfc14e8c700cdbd0a252655cbff4b4: 2023-03-05 19:55:49,380 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1678046144232.6fdfc14e8c700cdbd0a252655cbff4b4. 2023-03-05 19:55:49,380 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-05 19:55:49,760 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/f5738c13f081c7a8286ce16de3ec8a21/.tmp/family2/2c49c102359e467893c28e72bfbe9318 2023-03-05 19:55:49,763 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/f5738c13f081c7a8286ce16de3ec8a21/.tmp/family1/5eeef2442b714333896ff4ee143861ae as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/f5738c13f081c7a8286ce16de3ec8a21/family1/5eeef2442b714333896ff4ee143861ae 2023-03-05 19:55:49,766 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/f5738c13f081c7a8286ce16de3ec8a21/family1/5eeef2442b714333896ff4ee143861ae, entries=4, sequenceid=9, filesize=5.4 K 2023-03-05 19:55:49,767 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/f5738c13f081c7a8286ce16de3ec8a21/.tmp/family2/2c49c102359e467893c28e72bfbe9318 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/f5738c13f081c7a8286ce16de3ec8a21/family2/2c49c102359e467893c28e72bfbe9318 2023-03-05 19:55:49,769 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/f5738c13f081c7a8286ce16de3ec8a21/family2/2c49c102359e467893c28e72bfbe9318, entries=2, sequenceid=9, filesize=5.1 K 2023-03-05 19:55:49,770 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for f5738c13f081c7a8286ce16de3ec8a21 in 1644ms, sequenceid=9, compaction requested=false 2023-03-05 19:55:49,770 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for f5738c13f081c7a8286ce16de3ec8a21: 2023-03-05 19:55:49,770 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1678046144232.f5738c13f081c7a8286ce16de3ec8a21. 2023-03-05 19:55:49,770 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-05 19:55:49,770 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-05 19:55:49,770 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:49,770 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-03-05 19:55:49,770 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-03-05 19:55:49,773 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:49,773 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-05 19:55:49,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:49,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:55:49,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:55:49,773 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-05 19:55:49,773 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-05 19:55:49,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:55:49,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:55:49,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-05 19:55:49,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:49,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:55:49,774 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-03-05 19:55:49,774 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@63423010[Count = 0] remaining members to acquire global barrier 2023-03-05 19:55:49,775 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-03-05 19:55:49,775 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-05 19:55:49,776 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-05 19:55:49,776 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-05 19:55:49,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-05 19:55:49,776 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-03-05 19:55:49,776 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-03-05 19:55:49,776 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,37037,1678046018983' in zk 2023-03-05 19:55:49,776 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:49,776 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-05 19:55:49,784 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-03-05 19:55:49,784 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:49,784 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:55:49,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:49,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:55:49,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:55:49,784 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-03-05 19:55:49,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:55:49,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:55:49,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-05 19:55:49,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:49,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:55:49,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-05 19:55:49,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:49,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase4.apache.org,37037,1678046018983': 2023-03-05 19:55:49,786 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-03-05 19:55:49,786 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-03-05 19:55:49,786 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-05 19:55:49,786 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-05 19:55:49,786 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-03-05 19:55:49,786 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-05 19:55:49,790 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-05 19:55:49,790 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-05 19:55:49,791 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-05 19:55:49,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-05 19:55:49,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:55:49,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:55:49,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-05 19:55:49,791 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:55:49,791 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:55:49,791 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:49,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:55:49,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:55:49,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-05 19:55:49,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-05 19:55:49,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:55:49,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-05 19:55:49,792 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:49,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:49,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:55:49,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-05 19:55:49,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:49,794 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:55:49,794 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:49,794 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:55:49,794 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-03-05 19:55:49,794 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-05 19:55:49,794 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:55:49,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:55:49,794 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:55:49,794 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-05 19:55:49,794 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/ns:testMinorCompaction because node does not exist (not an error) 2023-03-05 19:55:49,794 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-03-05 19:55:49,795 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:55:49,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:55:49,794 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-05 19:55:49,795 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-05 19:55:49,795 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:49,795 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-05 19:55:49,795 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-05 19:55:49,795 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-05 19:55:49,895 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-05 19:55:49,896 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-05 19:55:49,897 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,,1678046144232.6fdfc14e8c700cdbd0a252655cbff4b4. with WAL disabled. Data may be lost in the event of a crash. 2023-03-05 19:55:49,897 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,1,1678046144232.f5738c13f081c7a8286ce16de3ec8a21. with WAL disabled. Data may be lost in the event of a crash. 2023-03-05 19:55:49,897 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,2,1678046144232.25ed4806e34b655e8718967079efb560. with WAL disabled. Data may be lost in the event of a crash. 2023-03-05 19:55:49,899 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-05 19:55:49,901 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(143): Procedure ns:testMinorCompaction was in running list but was completed. Accepting new attempt. 2023-03-05 19:55:49,901 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-03-05 19:55:49,901 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-03-05 19:55:49,901 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:55:49,901 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-03-05 19:55:49,901 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-03-05 19:55:49,901 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-05 19:55:49,901 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-05 19:55:49,903 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:49,903 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:55:49,903 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:55:49,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:55:49,903 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:49,903 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-05 19:55:49,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-05 19:55:49,903 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-05 19:55:49,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-05 19:55:49,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-05 19:55:49,904 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-03-05 19:55:49,904 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc ns:testMinorCompaction is still present, removing 2023-03-05 19:55:49,904 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-03-05 19:55:49,904 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-03-05 19:55:49,904 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:55:49,904 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-03-05 19:55:49,904 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1678046144232.25ed4806e34b655e8718967079efb560. 2023-03-05 19:55:49,904 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1678046144232.25ed4806e34b655e8718967079efb560. started... 2023-03-05 19:55:49,904 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-05 19:55:49,904 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-05 19:55:49,904 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1678046144232.6fdfc14e8c700cdbd0a252655cbff4b4. 2023-03-05 19:55:49,904 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1678046144232.f5738c13f081c7a8286ce16de3ec8a21. 2023-03-05 19:55:49,904 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1678046144232.6fdfc14e8c700cdbd0a252655cbff4b4. started... 2023-03-05 19:55:49,904 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1678046144232.f5738c13f081c7a8286ce16de3ec8a21. started... 2023-03-05 19:55:49,904 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 25ed4806e34b655e8718967079efb560 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-03-05 19:55:49,904 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 6fdfc14e8c700cdbd0a252655cbff4b4 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-03-05 19:55:49,904 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing f5738c13f081c7a8286ce16de3ec8a21 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-03-05 19:55:50,317 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230305c814bcf704574ff3a842caf642c86db7 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b20230305c814bcf704574ff3a842caf642c86db7 2023-03-05 19:55:50,320 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230305f833da723b4140ae95759b1652c9788b to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c20230305f833da723b4140ae95759b1652c9788b 2023-03-05 19:55:50,320 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230305bd4825abc6f74283b191e234e85780fc to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e20230305bd4825abc6f74283b191e234e85780fc 2023-03-05 19:55:50,724 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/f5738c13f081c7a8286ce16de3ec8a21/.tmp/family1/0958c8dcf48e4f4e8e03df3fef051884 2023-03-05 19:55:50,725 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/6fdfc14e8c700cdbd0a252655cbff4b4/.tmp/family1/dc01625027384c25a034d7c00743e9ba 2023-03-05 19:55:50,725 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/25ed4806e34b655e8718967079efb560/.tmp/family1/090b2a778f694f3fafed8a80eaaa3017 2023-03-05 19:55:50,877 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-05 19:55:51,138 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023030546b8cc631494426eb7b2c50cb8043bea to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b2023030546b8cc631494426eb7b2c50cb8043bea 2023-03-05 19:55:51,138 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230305a4c5cd7282d241d0895fe84c97682824 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c20230305a4c5cd7282d241d0895fe84c97682824 2023-03-05 19:55:51,138 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303055f768e10bfbc4ee8b9231d767e16c398 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e202303055f768e10bfbc4ee8b9231d767e16c398 2023-03-05 19:55:51,543 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/25ed4806e34b655e8718967079efb560/.tmp/family2/5bd67ab580a74b87a149212b3a3f675a 2023-03-05 19:55:51,543 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/f5738c13f081c7a8286ce16de3ec8a21/.tmp/family2/3ad0245181f940408262b7f15b527ade 2023-03-05 19:55:51,543 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/6fdfc14e8c700cdbd0a252655cbff4b4/.tmp/family2/90daa54ea7c74225bafe739c1089cfe4 2023-03-05 19:55:51,547 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/f5738c13f081c7a8286ce16de3ec8a21/.tmp/family1/0958c8dcf48e4f4e8e03df3fef051884 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/f5738c13f081c7a8286ce16de3ec8a21/family1/0958c8dcf48e4f4e8e03df3fef051884 2023-03-05 19:55:51,547 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/6fdfc14e8c700cdbd0a252655cbff4b4/.tmp/family1/dc01625027384c25a034d7c00743e9ba as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/6fdfc14e8c700cdbd0a252655cbff4b4/family1/dc01625027384c25a034d7c00743e9ba 2023-03-05 19:55:51,547 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/25ed4806e34b655e8718967079efb560/.tmp/family1/090b2a778f694f3fafed8a80eaaa3017 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/25ed4806e34b655e8718967079efb560/family1/090b2a778f694f3fafed8a80eaaa3017 2023-03-05 19:55:51,550 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/f5738c13f081c7a8286ce16de3ec8a21/family1/0958c8dcf48e4f4e8e03df3fef051884, entries=4, sequenceid=13, filesize=5.4 K 2023-03-05 19:55:51,550 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/6fdfc14e8c700cdbd0a252655cbff4b4/family1/dc01625027384c25a034d7c00743e9ba, entries=4, sequenceid=13, filesize=5.4 K 2023-03-05 19:55:51,551 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/f5738c13f081c7a8286ce16de3ec8a21/.tmp/family2/3ad0245181f940408262b7f15b527ade as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/f5738c13f081c7a8286ce16de3ec8a21/family2/3ad0245181f940408262b7f15b527ade 2023-03-05 19:55:51,551 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/25ed4806e34b655e8718967079efb560/family1/090b2a778f694f3fafed8a80eaaa3017, entries=4, sequenceid=13, filesize=5.4 K 2023-03-05 19:55:51,551 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/6fdfc14e8c700cdbd0a252655cbff4b4/.tmp/family2/90daa54ea7c74225bafe739c1089cfe4 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/6fdfc14e8c700cdbd0a252655cbff4b4/family2/90daa54ea7c74225bafe739c1089cfe4 2023-03-05 19:55:51,552 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/25ed4806e34b655e8718967079efb560/.tmp/family2/5bd67ab580a74b87a149212b3a3f675a as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/25ed4806e34b655e8718967079efb560/family2/5bd67ab580a74b87a149212b3a3f675a 2023-03-05 19:55:51,555 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/f5738c13f081c7a8286ce16de3ec8a21/family2/3ad0245181f940408262b7f15b527ade, entries=2, sequenceid=13, filesize=5.1 K 2023-03-05 19:55:51,555 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for f5738c13f081c7a8286ce16de3ec8a21 in 1651ms, sequenceid=13, compaction requested=true 2023-03-05 19:55:51,556 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for f5738c13f081c7a8286ce16de3ec8a21: 2023-03-05 19:55:51,556 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1678046144232.f5738c13f081c7a8286ce16de3ec8a21. 2023-03-05 19:55:51,556 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-05 19:55:51,556 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/6fdfc14e8c700cdbd0a252655cbff4b4/family2/90daa54ea7c74225bafe739c1089cfe4, entries=2, sequenceid=13, filesize=5.1 K 2023-03-05 19:55:51,557 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 6fdfc14e8c700cdbd0a252655cbff4b4 in 1653ms, sequenceid=13, compaction requested=true 2023-03-05 19:55:51,557 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 6fdfc14e8c700cdbd0a252655cbff4b4: 2023-03-05 19:55:51,557 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1678046144232.6fdfc14e8c700cdbd0a252655cbff4b4. 2023-03-05 19:55:51,557 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-05 19:55:51,561 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/25ed4806e34b655e8718967079efb560/family2/5bd67ab580a74b87a149212b3a3f675a, entries=2, sequenceid=13, filesize=5.1 K 2023-03-05 19:55:51,562 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 25ed4806e34b655e8718967079efb560 in 1658ms, sequenceid=13, compaction requested=true 2023-03-05 19:55:51,562 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 25ed4806e34b655e8718967079efb560: 2023-03-05 19:55:51,562 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1678046144232.25ed4806e34b655e8718967079efb560. 2023-03-05 19:55:51,562 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-05 19:55:51,562 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-05 19:55:51,562 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:51,562 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-03-05 19:55:51,562 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-03-05 19:55:51,564 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-05 19:55:51,564 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:51,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:51,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:55:51,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:55:51,564 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-05 19:55:51,564 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-05 19:55:51,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:55:51,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:55:51,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-05 19:55:51,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:51,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:55:51,565 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-03-05 19:55:51,565 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@596d5c1[Count = 0] remaining members to acquire global barrier 2023-03-05 19:55:51,565 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-03-05 19:55:51,565 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-05 19:55:51,566 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-05 19:55:51,566 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-05 19:55:51,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-05 19:55:51,566 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-03-05 19:55:51,566 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-03-05 19:55:51,567 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,37037,1678046018983' in zk 2023-03-05 19:55:51,566 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:51,567 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-05 19:55:51,568 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-03-05 19:55:51,568 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:51,568 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:55:51,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:51,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:55:51,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:55:51,568 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-03-05 19:55:51,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:55:51,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:55:51,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-05 19:55:51,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:51,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:55:51,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-05 19:55:51,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:51,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase4.apache.org,37037,1678046018983': 2023-03-05 19:55:51,570 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-03-05 19:55:51,570 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-03-05 19:55:51,570 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-05 19:55:51,570 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-05 19:55:51,570 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-03-05 19:55:51,570 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-05 19:55:51,573 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-05 19:55:51,573 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-05 19:55:51,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-05 19:55:51,573 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-05 19:55:51,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:55:51,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:55:51,573 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:55:51,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-05 19:55:51,573 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:51,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:55:51,573 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:55:51,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:55:51,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-05 19:55:51,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-05 19:55:51,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:55:51,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-05 19:55:51,574 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:51,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:51,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:55:51,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-05 19:55:51,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:51,576 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:51,577 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:55:51,577 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-05 19:55:51,577 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-03-05 19:55:51,577 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-05 19:55:51,577 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-05 19:55:51,577 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:55:51,577 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:55:51,577 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/ns:testMinorCompaction because node does not exist (not an error) 2023-03-05 19:55:51,577 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:55:51,577 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-03-05 19:55:51,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:55:51,577 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:51,577 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-05 19:55:51,577 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-05 19:55:51,577 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-05 19:55:51,577 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-05 19:55:51,577 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:55:51,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:55:51,677 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-05 19:55:51,678 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-05 19:55:51,679 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,,1678046144232.6fdfc14e8c700cdbd0a252655cbff4b4. with WAL disabled. Data may be lost in the event of a crash. 2023-03-05 19:55:51,679 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,1,1678046144232.f5738c13f081c7a8286ce16de3ec8a21. with WAL disabled. Data may be lost in the event of a crash. 2023-03-05 19:55:51,679 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,2,1678046144232.25ed4806e34b655e8718967079efb560. with WAL disabled. Data may be lost in the event of a crash. 2023-03-05 19:55:51,681 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-05 19:55:51,683 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(143): Procedure ns:testMinorCompaction was in running list but was completed. Accepting new attempt. 2023-03-05 19:55:51,683 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-03-05 19:55:51,683 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-03-05 19:55:51,683 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:55:51,683 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-03-05 19:55:51,683 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-03-05 19:55:51,683 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-05 19:55:51,683 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-05 19:55:51,685 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:55:51,685 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:51,685 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:55:51,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:55:51,685 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:51,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-05 19:55:51,685 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-05 19:55:51,685 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-05 19:55:51,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-05 19:55:51,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-05 19:55:51,686 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-03-05 19:55:51,686 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc ns:testMinorCompaction is still present, removing 2023-03-05 19:55:51,686 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-03-05 19:55:51,686 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-03-05 19:55:51,686 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:55:51,686 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-03-05 19:55:51,686 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1678046144232.25ed4806e34b655e8718967079efb560. 2023-03-05 19:55:51,686 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-05 19:55:51,686 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-05 19:55:51,686 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1678046144232.25ed4806e34b655e8718967079efb560. started... 2023-03-05 19:55:51,686 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1678046144232.f5738c13f081c7a8286ce16de3ec8a21. 2023-03-05 19:55:51,686 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1678046144232.6fdfc14e8c700cdbd0a252655cbff4b4. 2023-03-05 19:55:51,686 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 25ed4806e34b655e8718967079efb560 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-03-05 19:55:51,686 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1678046144232.f5738c13f081c7a8286ce16de3ec8a21. started... 2023-03-05 19:55:51,686 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1678046144232.6fdfc14e8c700cdbd0a252655cbff4b4. started... 2023-03-05 19:55:51,686 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing f5738c13f081c7a8286ce16de3ec8a21 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-03-05 19:55:51,687 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 6fdfc14e8c700cdbd0a252655cbff4b4 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-03-05 19:55:52,098 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303056e1afe76118c4f4f92244036ebf95eca to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b202303056e1afe76118c4f4f92244036ebf95eca 2023-03-05 19:55:52,098 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230305be90b228e5a94d8c9ccdd5af18d6e90e to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c20230305be90b228e5a94d8c9ccdd5af18d6e90e 2023-03-05 19:55:52,098 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230305be3cd65f27e543c08467a831bd6648d8 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e20230305be3cd65f27e543c08467a831bd6648d8 2023-03-05 19:55:52,503 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/f5738c13f081c7a8286ce16de3ec8a21/.tmp/family1/6efca904c2cb4ca58e88141495470acf 2023-03-05 19:55:52,503 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/25ed4806e34b655e8718967079efb560/.tmp/family1/0be3824ff3004d10aa36709483a195a8 2023-03-05 19:55:52,503 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/6fdfc14e8c700cdbd0a252655cbff4b4/.tmp/family1/5178bc46025e4fa4ae8453540ec38ce2 2023-03-05 19:55:52,917 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230305a3efd42ae2944683ae5e4a9718fcd3d8 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c20230305a3efd42ae2944683ae5e4a9718fcd3d8 2023-03-05 19:55:52,917 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303054186bfc4561b42609e5ce312876107a2 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e202303054186bfc4561b42609e5ce312876107a2 2023-03-05 19:55:52,917 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230305bbf09c33a5284a2a8060e90a06dd5b86 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b20230305bbf09c33a5284a2a8060e90a06dd5b86 2023-03-05 19:55:53,323 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/25ed4806e34b655e8718967079efb560/.tmp/family2/447232a95a744dc5aef9402a748ca6a6 2023-03-05 19:55:53,323 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/f5738c13f081c7a8286ce16de3ec8a21/.tmp/family2/024df8f7ffcf44c4843221320fdacf20 2023-03-05 19:55:53,324 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/6fdfc14e8c700cdbd0a252655cbff4b4/.tmp/family2/f43387a93a0e4a619d881dc70625ee18 2023-03-05 19:55:53,327 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/6fdfc14e8c700cdbd0a252655cbff4b4/.tmp/family1/5178bc46025e4fa4ae8453540ec38ce2 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/6fdfc14e8c700cdbd0a252655cbff4b4/family1/5178bc46025e4fa4ae8453540ec38ce2 2023-03-05 19:55:53,327 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/25ed4806e34b655e8718967079efb560/.tmp/family1/0be3824ff3004d10aa36709483a195a8 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/25ed4806e34b655e8718967079efb560/family1/0be3824ff3004d10aa36709483a195a8 2023-03-05 19:55:53,327 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/f5738c13f081c7a8286ce16de3ec8a21/.tmp/family1/6efca904c2cb4ca58e88141495470acf as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/f5738c13f081c7a8286ce16de3ec8a21/family1/6efca904c2cb4ca58e88141495470acf 2023-03-05 19:55:53,331 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/6fdfc14e8c700cdbd0a252655cbff4b4/family1/5178bc46025e4fa4ae8453540ec38ce2, entries=4, sequenceid=17, filesize=5.4 K 2023-03-05 19:55:53,331 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/25ed4806e34b655e8718967079efb560/family1/0be3824ff3004d10aa36709483a195a8, entries=4, sequenceid=17, filesize=5.4 K 2023-03-05 19:55:53,331 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/f5738c13f081c7a8286ce16de3ec8a21/family1/6efca904c2cb4ca58e88141495470acf, entries=4, sequenceid=17, filesize=5.4 K 2023-03-05 19:55:53,332 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/6fdfc14e8c700cdbd0a252655cbff4b4/.tmp/family2/f43387a93a0e4a619d881dc70625ee18 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/6fdfc14e8c700cdbd0a252655cbff4b4/family2/f43387a93a0e4a619d881dc70625ee18 2023-03-05 19:55:53,332 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/25ed4806e34b655e8718967079efb560/.tmp/family2/447232a95a744dc5aef9402a748ca6a6 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/25ed4806e34b655e8718967079efb560/family2/447232a95a744dc5aef9402a748ca6a6 2023-03-05 19:55:53,332 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/f5738c13f081c7a8286ce16de3ec8a21/.tmp/family2/024df8f7ffcf44c4843221320fdacf20 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/f5738c13f081c7a8286ce16de3ec8a21/family2/024df8f7ffcf44c4843221320fdacf20 2023-03-05 19:55:53,335 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/6fdfc14e8c700cdbd0a252655cbff4b4/family2/f43387a93a0e4a619d881dc70625ee18, entries=2, sequenceid=17, filesize=5.1 K 2023-03-05 19:55:53,336 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/25ed4806e34b655e8718967079efb560/family2/447232a95a744dc5aef9402a748ca6a6, entries=2, sequenceid=17, filesize=5.1 K 2023-03-05 19:55:53,336 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/f5738c13f081c7a8286ce16de3ec8a21/family2/024df8f7ffcf44c4843221320fdacf20, entries=2, sequenceid=17, filesize=5.1 K 2023-03-05 19:55:53,336 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 6fdfc14e8c700cdbd0a252655cbff4b4 in 1650ms, sequenceid=17, compaction requested=true 2023-03-05 19:55:53,336 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 25ed4806e34b655e8718967079efb560 in 1650ms, sequenceid=17, compaction requested=true 2023-03-05 19:55:53,336 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 6fdfc14e8c700cdbd0a252655cbff4b4: 2023-03-05 19:55:53,336 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 25ed4806e34b655e8718967079efb560: 2023-03-05 19:55:53,336 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for f5738c13f081c7a8286ce16de3ec8a21 in 1650ms, sequenceid=17, compaction requested=true 2023-03-05 19:55:53,336 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1678046144232.25ed4806e34b655e8718967079efb560. 2023-03-05 19:55:53,336 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for f5738c13f081c7a8286ce16de3ec8a21: 2023-03-05 19:55:53,336 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1678046144232.f5738c13f081c7a8286ce16de3ec8a21. 2023-03-05 19:55:53,336 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-05 19:55:53,336 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1678046144232.6fdfc14e8c700cdbd0a252655cbff4b4. 2023-03-05 19:55:53,337 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-05 19:55:53,337 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-05 19:55:53,337 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-05 19:55:53,337 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:53,337 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-03-05 19:55:53,337 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-03-05 19:55:53,338 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-05 19:55:53,338 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:53,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:53,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:55:53,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:55:53,338 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-05 19:55:53,338 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-05 19:55:53,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:55:53,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:55:53,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-05 19:55:53,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:53,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:55:53,339 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-03-05 19:55:53,339 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@25796909[Count = 0] remaining members to acquire global barrier 2023-03-05 19:55:53,339 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-03-05 19:55:53,339 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-05 19:55:53,341 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-05 19:55:53,341 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-05 19:55:53,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-05 19:55:53,342 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-03-05 19:55:53,342 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-03-05 19:55:53,342 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,37037,1678046018983' in zk 2023-03-05 19:55:53,342 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:53,342 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-05 19:55:53,343 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:53,343 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-03-05 19:55:53,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:53,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:55:53,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:55:53,343 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:55:53,343 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-03-05 19:55:53,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:55:53,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:55:53,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-05 19:55:53,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:53,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:55:53,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-05 19:55:53,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:53,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase4.apache.org,37037,1678046018983': 2023-03-05 19:55:53,345 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-03-05 19:55:53,345 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-03-05 19:55:53,345 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-05 19:55:53,345 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-05 19:55:53,345 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-03-05 19:55:53,345 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-05 19:55:53,346 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-05 19:55:53,346 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-05 19:55:53,346 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:55:53,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-05 19:55:53,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:55:53,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:55:53,346 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-05 19:55:53,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-05 19:55:53,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:55:53,346 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:53,346 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:55:53,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:55:53,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-05 19:55:53,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-05 19:55:53,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:55:53,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-05 19:55:53,347 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:53,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:53,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:55:53,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-05 19:55:53,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:53,349 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:53,349 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-05 19:55:53,349 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-05 19:55:53,349 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:53,350 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-05 19:55:53,350 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-05 19:55:53,350 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:55:53,350 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-05 19:55:53,349 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:55:53,350 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-03-05 19:55:53,350 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-05 19:55:53,350 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:55:53,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:55:53,350 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:55:53,350 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-03-05 19:55:53,350 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-05 19:55:53,350 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:55:53,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:55:53,450 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-05 19:55:53,451 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-05 19:55:53,459 DEBUG [Listener at localhost/40787] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-05 19:55:53,466 DEBUG [Listener at localhost/40787] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-05 19:55:53,473 DEBUG [Listener at localhost/40787] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-05 19:55:53,474 DEBUG [Listener at localhost/40787] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-05 19:55:53,500 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-05 19:55:53,502 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(143): Procedure ns:testMinorCompaction was in running list but was completed. Accepting new attempt. 2023-03-05 19:55:53,502 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-03-05 19:55:53,502 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-03-05 19:55:53,502 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:55:53,502 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-03-05 19:55:53,502 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-03-05 19:55:53,503 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-05 19:55:53,503 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-05 19:55:53,505 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:53,505 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:55:53,505 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:55:53,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:55:53,505 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:53,505 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-05 19:55:53,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-05 19:55:53,505 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-05 19:55:53,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-05 19:55:53,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-05 19:55:53,506 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-03-05 19:55:53,506 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc ns:testMinorCompaction is still present, removing 2023-03-05 19:55:53,506 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-03-05 19:55:53,506 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-03-05 19:55:53,506 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:55:53,506 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-03-05 19:55:53,506 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1678046144232.25ed4806e34b655e8718967079efb560. 2023-03-05 19:55:53,506 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1678046144232.25ed4806e34b655e8718967079efb560. started... 2023-03-05 19:55:53,506 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1678046144232.f5738c13f081c7a8286ce16de3ec8a21. 2023-03-05 19:55:53,507 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1678046144232.f5738c13f081c7a8286ce16de3ec8a21. started... 2023-03-05 19:55:53,507 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 25ed4806e34b655e8718967079efb560 2/2 column families, dataSize=138 B heapSize=944 B 2023-03-05 19:55:53,507 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing f5738c13f081c7a8286ce16de3ec8a21 2/2 column families, dataSize=138 B heapSize=944 B 2023-03-05 19:55:53,507 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-05 19:55:53,507 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-05 19:55:53,507 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1678046144232.6fdfc14e8c700cdbd0a252655cbff4b4. 2023-03-05 19:55:53,507 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1678046144232.6fdfc14e8c700cdbd0a252655cbff4b4. started... 2023-03-05 19:55:53,507 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 6fdfc14e8c700cdbd0a252655cbff4b4 2/2 column families, dataSize=138 B heapSize=944 B 2023-03-05 19:55:53,532 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/6fdfc14e8c700cdbd0a252655cbff4b4/.tmp/family1/629486c6ac1f4c33be772dfa9f232be7 2023-03-05 19:55:53,535 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 629486c6ac1f4c33be772dfa9f232be7 2023-03-05 19:55:53,946 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/6fdfc14e8c700cdbd0a252655cbff4b4/.tmp/family2/d0040a46052e415597bf954de7cb6c4c 2023-03-05 19:55:53,949 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d0040a46052e415597bf954de7cb6c4c 2023-03-05 19:55:53,949 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/6fdfc14e8c700cdbd0a252655cbff4b4/.tmp/family1/629486c6ac1f4c33be772dfa9f232be7 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/6fdfc14e8c700cdbd0a252655cbff4b4/family1/629486c6ac1f4c33be772dfa9f232be7 2023-03-05 19:55:53,952 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 629486c6ac1f4c33be772dfa9f232be7 2023-03-05 19:55:53,952 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/6fdfc14e8c700cdbd0a252655cbff4b4/family1/629486c6ac1f4c33be772dfa9f232be7, entries=3, sequenceid=23, filesize=5.0 K 2023-03-05 19:55:53,953 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/6fdfc14e8c700cdbd0a252655cbff4b4/.tmp/family2/d0040a46052e415597bf954de7cb6c4c as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/6fdfc14e8c700cdbd0a252655cbff4b4/family2/d0040a46052e415597bf954de7cb6c4c 2023-03-05 19:55:53,956 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d0040a46052e415597bf954de7cb6c4c 2023-03-05 19:55:53,956 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/6fdfc14e8c700cdbd0a252655cbff4b4/family2/d0040a46052e415597bf954de7cb6c4c, entries=1, sequenceid=23, filesize=5.0 K 2023-03-05 19:55:53,956 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 6fdfc14e8c700cdbd0a252655cbff4b4 in 449ms, sequenceid=23, compaction requested=true 2023-03-05 19:55:53,956 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 6fdfc14e8c700cdbd0a252655cbff4b4: 2023-03-05 19:55:53,957 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1678046144232.6fdfc14e8c700cdbd0a252655cbff4b4. 2023-03-05 19:55:53,957 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-05 19:55:54,334 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/25ed4806e34b655e8718967079efb560/.tmp/family1/334ebde95ca14c7b86a67b088524e136 2023-03-05 19:55:54,334 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/f5738c13f081c7a8286ce16de3ec8a21/.tmp/family1/f867d36a504141ab970cb77201a03c1f 2023-03-05 19:55:54,337 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 334ebde95ca14c7b86a67b088524e136 2023-03-05 19:55:54,337 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f867d36a504141ab970cb77201a03c1f 2023-03-05 19:55:55,148 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/f5738c13f081c7a8286ce16de3ec8a21/.tmp/family2/0b51cf69b1fe409f9cac25d3afa196cc 2023-03-05 19:55:55,148 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/25ed4806e34b655e8718967079efb560/.tmp/family2/eba64a3519c14db382cfd7601362545b 2023-03-05 19:55:55,153 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0b51cf69b1fe409f9cac25d3afa196cc 2023-03-05 19:55:55,154 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/f5738c13f081c7a8286ce16de3ec8a21/.tmp/family1/f867d36a504141ab970cb77201a03c1f as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/f5738c13f081c7a8286ce16de3ec8a21/family1/f867d36a504141ab970cb77201a03c1f 2023-03-05 19:55:55,157 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for eba64a3519c14db382cfd7601362545b 2023-03-05 19:55:55,158 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f867d36a504141ab970cb77201a03c1f 2023-03-05 19:55:55,158 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/25ed4806e34b655e8718967079efb560/.tmp/family1/334ebde95ca14c7b86a67b088524e136 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/25ed4806e34b655e8718967079efb560/family1/334ebde95ca14c7b86a67b088524e136 2023-03-05 19:55:55,158 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/f5738c13f081c7a8286ce16de3ec8a21/family1/f867d36a504141ab970cb77201a03c1f, entries=3, sequenceid=23, filesize=5.0 K 2023-03-05 19:55:55,159 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/f5738c13f081c7a8286ce16de3ec8a21/.tmp/family2/0b51cf69b1fe409f9cac25d3afa196cc as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/f5738c13f081c7a8286ce16de3ec8a21/family2/0b51cf69b1fe409f9cac25d3afa196cc 2023-03-05 19:55:55,161 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 334ebde95ca14c7b86a67b088524e136 2023-03-05 19:55:55,161 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/25ed4806e34b655e8718967079efb560/family1/334ebde95ca14c7b86a67b088524e136, entries=3, sequenceid=23, filesize=5.0 K 2023-03-05 19:55:55,162 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/25ed4806e34b655e8718967079efb560/.tmp/family2/eba64a3519c14db382cfd7601362545b as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/25ed4806e34b655e8718967079efb560/family2/eba64a3519c14db382cfd7601362545b 2023-03-05 19:55:55,163 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0b51cf69b1fe409f9cac25d3afa196cc 2023-03-05 19:55:55,163 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/f5738c13f081c7a8286ce16de3ec8a21/family2/0b51cf69b1fe409f9cac25d3afa196cc, entries=1, sequenceid=23, filesize=5.0 K 2023-03-05 19:55:55,164 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for f5738c13f081c7a8286ce16de3ec8a21 in 1657ms, sequenceid=23, compaction requested=true 2023-03-05 19:55:55,164 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for f5738c13f081c7a8286ce16de3ec8a21: 2023-03-05 19:55:55,164 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1678046144232.f5738c13f081c7a8286ce16de3ec8a21. 2023-03-05 19:55:55,164 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-05 19:55:55,165 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for eba64a3519c14db382cfd7601362545b 2023-03-05 19:55:55,166 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/25ed4806e34b655e8718967079efb560/family2/eba64a3519c14db382cfd7601362545b, entries=1, sequenceid=23, filesize=5.0 K 2023-03-05 19:55:55,166 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 25ed4806e34b655e8718967079efb560 in 1660ms, sequenceid=23, compaction requested=true 2023-03-05 19:55:55,166 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 25ed4806e34b655e8718967079efb560: 2023-03-05 19:55:55,166 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1678046144232.25ed4806e34b655e8718967079efb560. 2023-03-05 19:55:55,166 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-05 19:55:55,166 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-05 19:55:55,166 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:55,166 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-03-05 19:55:55,166 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-03-05 19:55:55,168 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-05 19:55:55,168 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:55,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:55,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:55:55,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:55:55,169 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-05 19:55:55,169 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-05 19:55:55,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:55:55,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:55:55,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-05 19:55:55,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:55,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:55:55,171 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-03-05 19:55:55,171 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4b585e9[Count = 0] remaining members to acquire global barrier 2023-03-05 19:55:55,171 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-03-05 19:55:55,171 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-05 19:55:55,172 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-05 19:55:55,172 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-05 19:55:55,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-05 19:55:55,172 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-03-05 19:55:55,172 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-03-05 19:55:55,172 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,37037,1678046018983' in zk 2023-03-05 19:55:55,172 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:55,172 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-05 19:55:55,174 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-03-05 19:55:55,174 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:55:55,174 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:55,174 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-03-05 19:55:55,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:55,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:55:55,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:55:55,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:55:55,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:55:55,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-05 19:55:55,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:55,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:55:55,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-05 19:55:55,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:55,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase4.apache.org,37037,1678046018983': 2023-03-05 19:55:55,176 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-03-05 19:55:55,176 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-03-05 19:55:55,176 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-05 19:55:55,176 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-05 19:55:55,176 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-03-05 19:55:55,177 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-05 19:55:55,182 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-05 19:55:55,182 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-05 19:55:55,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-05 19:55:55,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:55:55,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:55:55,182 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-05 19:55:55,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-05 19:55:55,182 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:55:55,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:55:55,182 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:55:55,182 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:55,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:55:55,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-05 19:55:55,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-05 19:55:55,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:55:55,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-05 19:55:55,184 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:55,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:55,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:55:55,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-05 19:55:55,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:55,186 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:55,186 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-05 19:55:55,186 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-05 19:55:55,186 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:55:55,186 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-03-05 19:55:55,186 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-05 19:55:55,186 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:55:55,187 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:55:55,186 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-05 19:55:55,187 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-05 19:55:55,187 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-05 19:55:55,186 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:55:55,187 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-03-05 19:55:55,187 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:55:55,187 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-05 19:55:55,187 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/ns:testMinorCompaction because node does not exist (not an error) 2023-03-05 19:55:55,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:55:55,187 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:55:55,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:55:55,287 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-05 19:55:55,288 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-05 19:55:55,288 DEBUG [Listener at localhost/40787] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-05 19:55:55,289 DEBUG [Listener at localhost/40787] regionserver.HStore(1912): 25ed4806e34b655e8718967079efb560/family1 is initiating major compaction (all files) 2023-03-05 19:55:55,289 INFO [Listener at localhost/40787] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-05 19:55:55,289 INFO [Listener at localhost/40787] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-05 19:55:55,289 INFO [Listener at localhost/40787] regionserver.HRegion(2259): Starting compaction of 25ed4806e34b655e8718967079efb560/family1 in ns:testMinorCompaction,2,1678046144232.25ed4806e34b655e8718967079efb560. 2023-03-05 19:55:55,289 INFO [Listener at localhost/40787] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/25ed4806e34b655e8718967079efb560/family1/1c5ca09a26864f08a5625671975ef036, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/25ed4806e34b655e8718967079efb560/family1/959ce2e3e340495cbf38b70641ea0a02, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/25ed4806e34b655e8718967079efb560/family1/090b2a778f694f3fafed8a80eaaa3017, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/25ed4806e34b655e8718967079efb560/family1/0be3824ff3004d10aa36709483a195a8, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/25ed4806e34b655e8718967079efb560/family1/334ebde95ca14c7b86a67b088524e136] into tmpdir=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/25ed4806e34b655e8718967079efb560/.tmp, totalSize=26.6 K 2023-03-05 19:55:55,290 DEBUG [Listener at localhost/40787] compactions.Compactor(207): Compacting 1c5ca09a26864f08a5625671975ef036, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678046146347 2023-03-05 19:55:55,291 DEBUG [Listener at localhost/40787] compactions.Compactor(207): Compacting 959ce2e3e340495cbf38b70641ea0a02, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678046148119 2023-03-05 19:55:55,291 DEBUG [Listener at localhost/40787] compactions.Compactor(207): Compacting 090b2a778f694f3fafed8a80eaaa3017, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678046149897 2023-03-05 19:55:55,292 DEBUG [Listener at localhost/40787] compactions.Compactor(207): Compacting 0be3824ff3004d10aa36709483a195a8, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678046151679 2023-03-05 19:55:55,292 DEBUG [Listener at localhost/40787] compactions.Compactor(207): Compacting 334ebde95ca14c7b86a67b088524e136, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-05 19:55:55,305 INFO [Listener at localhost/40787] throttle.PressureAwareThroughputController(145): 25ed4806e34b655e8718967079efb560#family1#compaction#149 average throughput is 0.41 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-05 19:55:55,717 INFO [Listener at localhost/40787] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230305e61d8371ebb94064a8df627c46b77a99_del 2023-03-05 19:55:55,717 INFO [Listener at localhost/40787] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230305e61d8371ebb94064a8df627c46b77a99_del to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c20230305e61d8371ebb94064a8df627c46b77a99_del 2023-03-05 19:55:56,130 INFO [Listener at localhost/40787] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for bf33089b1e1e4578875cb07dfe77f5ae 2023-03-05 19:55:56,131 DEBUG [Listener at localhost/40787] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/25ed4806e34b655e8718967079efb560/.tmp/family1/bf33089b1e1e4578875cb07dfe77f5ae as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/25ed4806e34b655e8718967079efb560/family1/bf33089b1e1e4578875cb07dfe77f5ae 2023-03-05 19:55:56,135 INFO [Listener at localhost/40787] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for bf33089b1e1e4578875cb07dfe77f5ae 2023-03-05 19:55:56,136 INFO [Listener at localhost/40787] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 25ed4806e34b655e8718967079efb560/family1 of 25ed4806e34b655e8718967079efb560 into bf33089b1e1e4578875cb07dfe77f5ae(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-05 19:55:56,136 DEBUG [Listener at localhost/40787] regionserver.HRegion(2289): Compaction status journal for 25ed4806e34b655e8718967079efb560: 2023-03-05 19:55:56,136 DEBUG [Listener at localhost/40787] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-05 19:55:56,136 DEBUG [Listener at localhost/40787] regionserver.HStore(1912): 25ed4806e34b655e8718967079efb560/family2 is initiating major compaction (all files) 2023-03-05 19:55:56,136 INFO [Listener at localhost/40787] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-05 19:55:56,136 INFO [Listener at localhost/40787] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-05 19:55:56,137 INFO [Listener at localhost/40787] regionserver.HRegion(2259): Starting compaction of 25ed4806e34b655e8718967079efb560/family2 in ns:testMinorCompaction,2,1678046144232.25ed4806e34b655e8718967079efb560. 2023-03-05 19:55:56,137 INFO [Listener at localhost/40787] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/25ed4806e34b655e8718967079efb560/family2/a4dcc57fdaee455db32e516ed78bb0bc, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/25ed4806e34b655e8718967079efb560/family2/7168e07305da425b895922ef287d801c, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/25ed4806e34b655e8718967079efb560/family2/5bd67ab580a74b87a149212b3a3f675a, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/25ed4806e34b655e8718967079efb560/family2/447232a95a744dc5aef9402a748ca6a6, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/25ed4806e34b655e8718967079efb560/family2/eba64a3519c14db382cfd7601362545b] into tmpdir=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/25ed4806e34b655e8718967079efb560/.tmp, totalSize=25.3 K 2023-03-05 19:55:56,138 DEBUG [Listener at localhost/40787] compactions.Compactor(207): Compacting a4dcc57fdaee455db32e516ed78bb0bc, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678046146347 2023-03-05 19:55:56,138 DEBUG [Listener at localhost/40787] compactions.Compactor(207): Compacting 7168e07305da425b895922ef287d801c, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678046148119 2023-03-05 19:55:56,139 DEBUG [Listener at localhost/40787] compactions.Compactor(207): Compacting 5bd67ab580a74b87a149212b3a3f675a, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678046149897 2023-03-05 19:55:56,139 DEBUG [Listener at localhost/40787] compactions.Compactor(207): Compacting 447232a95a744dc5aef9402a748ca6a6, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678046151679 2023-03-05 19:55:56,139 DEBUG [Listener at localhost/40787] compactions.Compactor(207): Compacting eba64a3519c14db382cfd7601362545b, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-05 19:55:56,151 INFO [Listener at localhost/40787] throttle.PressureAwareThroughputController(145): 25ed4806e34b655e8718967079efb560#family2#compaction#150 average throughput is 0.25 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-05 19:55:56,180 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/25ed4806e34b655e8718967079efb560/family1/1c5ca09a26864f08a5625671975ef036, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/25ed4806e34b655e8718967079efb560/family1/959ce2e3e340495cbf38b70641ea0a02, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/25ed4806e34b655e8718967079efb560/family1/090b2a778f694f3fafed8a80eaaa3017, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/25ed4806e34b655e8718967079efb560/family1/0be3824ff3004d10aa36709483a195a8, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/25ed4806e34b655e8718967079efb560/family1/334ebde95ca14c7b86a67b088524e136] to archive 2023-03-05 19:55:56,182 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-05 19:55:56,184 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/25ed4806e34b655e8718967079efb560/family1/1c5ca09a26864f08a5625671975ef036 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/ns/testMinorCompaction/25ed4806e34b655e8718967079efb560/family1/1c5ca09a26864f08a5625671975ef036 2023-03-05 19:55:56,185 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/25ed4806e34b655e8718967079efb560/family1/959ce2e3e340495cbf38b70641ea0a02 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/ns/testMinorCompaction/25ed4806e34b655e8718967079efb560/family1/959ce2e3e340495cbf38b70641ea0a02 2023-03-05 19:55:56,186 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/25ed4806e34b655e8718967079efb560/family1/090b2a778f694f3fafed8a80eaaa3017 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/ns/testMinorCompaction/25ed4806e34b655e8718967079efb560/family1/090b2a778f694f3fafed8a80eaaa3017 2023-03-05 19:55:56,187 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/25ed4806e34b655e8718967079efb560/family1/0be3824ff3004d10aa36709483a195a8 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/ns/testMinorCompaction/25ed4806e34b655e8718967079efb560/family1/0be3824ff3004d10aa36709483a195a8 2023-03-05 19:55:56,188 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/25ed4806e34b655e8718967079efb560/family1/334ebde95ca14c7b86a67b088524e136 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/ns/testMinorCompaction/25ed4806e34b655e8718967079efb560/family1/334ebde95ca14c7b86a67b088524e136 2023-03-05 19:55:56,562 INFO [Listener at localhost/40787] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230305cd19c1a8d9ac4f659c65a965d7dcf6fe_del 2023-03-05 19:55:56,562 INFO [Listener at localhost/40787] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230305cd19c1a8d9ac4f659c65a965d7dcf6fe_del to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c20230305cd19c1a8d9ac4f659c65a965d7dcf6fe_del 2023-03-05 19:55:56,573 INFO [Listener at localhost/40787] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for dedfc1fceb6a49c784582ea4df27a361 2023-03-05 19:55:56,574 DEBUG [Listener at localhost/40787] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/25ed4806e34b655e8718967079efb560/.tmp/family2/dedfc1fceb6a49c784582ea4df27a361 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/25ed4806e34b655e8718967079efb560/family2/dedfc1fceb6a49c784582ea4df27a361 2023-03-05 19:55:56,577 INFO [Listener at localhost/40787] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for dedfc1fceb6a49c784582ea4df27a361 2023-03-05 19:55:56,578 INFO [Listener at localhost/40787] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 25ed4806e34b655e8718967079efb560/family2 of 25ed4806e34b655e8718967079efb560 into dedfc1fceb6a49c784582ea4df27a361(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-05 19:55:56,578 DEBUG [Listener at localhost/40787] regionserver.HRegion(2289): Compaction status journal for 25ed4806e34b655e8718967079efb560: 2023-03-05 19:55:56,578 DEBUG [Listener at localhost/40787] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-05 19:55:56,578 DEBUG [Listener at localhost/40787] regionserver.HStore(1912): f5738c13f081c7a8286ce16de3ec8a21/family1 is initiating major compaction (all files) 2023-03-05 19:55:56,578 INFO [Listener at localhost/40787] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-05 19:55:56,578 INFO [Listener at localhost/40787] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-05 19:55:56,579 INFO [Listener at localhost/40787] regionserver.HRegion(2259): Starting compaction of f5738c13f081c7a8286ce16de3ec8a21/family1 in ns:testMinorCompaction,1,1678046144232.f5738c13f081c7a8286ce16de3ec8a21. 2023-03-05 19:55:56,579 INFO [Listener at localhost/40787] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/f5738c13f081c7a8286ce16de3ec8a21/family1/7ea37ba56aeb4d21a21a5d3bf889223c, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/f5738c13f081c7a8286ce16de3ec8a21/family1/5eeef2442b714333896ff4ee143861ae, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/f5738c13f081c7a8286ce16de3ec8a21/family1/0958c8dcf48e4f4e8e03df3fef051884, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/f5738c13f081c7a8286ce16de3ec8a21/family1/6efca904c2cb4ca58e88141495470acf, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/f5738c13f081c7a8286ce16de3ec8a21/family1/f867d36a504141ab970cb77201a03c1f] into tmpdir=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/f5738c13f081c7a8286ce16de3ec8a21/.tmp, totalSize=26.6 K 2023-03-05 19:55:56,580 DEBUG [Listener at localhost/40787] compactions.Compactor(207): Compacting 7ea37ba56aeb4d21a21a5d3bf889223c, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678046146346 2023-03-05 19:55:56,580 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/25ed4806e34b655e8718967079efb560/family2/a4dcc57fdaee455db32e516ed78bb0bc, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/25ed4806e34b655e8718967079efb560/family2/7168e07305da425b895922ef287d801c, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/25ed4806e34b655e8718967079efb560/family2/5bd67ab580a74b87a149212b3a3f675a, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/25ed4806e34b655e8718967079efb560/family2/447232a95a744dc5aef9402a748ca6a6, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/25ed4806e34b655e8718967079efb560/family2/eba64a3519c14db382cfd7601362545b] to archive 2023-03-05 19:55:56,580 DEBUG [Listener at localhost/40787] compactions.Compactor(207): Compacting 5eeef2442b714333896ff4ee143861ae, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678046148119 2023-03-05 19:55:56,581 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-05 19:55:56,581 DEBUG [Listener at localhost/40787] compactions.Compactor(207): Compacting 0958c8dcf48e4f4e8e03df3fef051884, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678046149897 2023-03-05 19:55:56,581 DEBUG [Listener at localhost/40787] compactions.Compactor(207): Compacting 6efca904c2cb4ca58e88141495470acf, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678046151679 2023-03-05 19:55:56,581 DEBUG [Listener at localhost/40787] compactions.Compactor(207): Compacting f867d36a504141ab970cb77201a03c1f, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-05 19:55:56,582 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/25ed4806e34b655e8718967079efb560/family2/a4dcc57fdaee455db32e516ed78bb0bc to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/ns/testMinorCompaction/25ed4806e34b655e8718967079efb560/family2/a4dcc57fdaee455db32e516ed78bb0bc 2023-03-05 19:55:56,584 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/25ed4806e34b655e8718967079efb560/family2/7168e07305da425b895922ef287d801c to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/ns/testMinorCompaction/25ed4806e34b655e8718967079efb560/family2/7168e07305da425b895922ef287d801c 2023-03-05 19:55:56,585 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/25ed4806e34b655e8718967079efb560/family2/5bd67ab580a74b87a149212b3a3f675a to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/ns/testMinorCompaction/25ed4806e34b655e8718967079efb560/family2/5bd67ab580a74b87a149212b3a3f675a 2023-03-05 19:55:56,586 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/25ed4806e34b655e8718967079efb560/family2/447232a95a744dc5aef9402a748ca6a6 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/ns/testMinorCompaction/25ed4806e34b655e8718967079efb560/family2/447232a95a744dc5aef9402a748ca6a6 2023-03-05 19:55:56,587 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/25ed4806e34b655e8718967079efb560/family2/eba64a3519c14db382cfd7601362545b to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/ns/testMinorCompaction/25ed4806e34b655e8718967079efb560/family2/eba64a3519c14db382cfd7601362545b 2023-03-05 19:55:56,590 INFO [Listener at localhost/40787] throttle.PressureAwareThroughputController(145): f5738c13f081c7a8286ce16de3ec8a21#family1#compaction#151 average throughput is 0.83 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-05 19:55:56,999 INFO [Listener at localhost/40787] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b202303051654ba92a3414768a48ff37a27675159_del 2023-03-05 19:55:57,000 INFO [Listener at localhost/40787] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303051654ba92a3414768a48ff37a27675159_del to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b202303051654ba92a3414768a48ff37a27675159_del 2023-03-05 19:55:57,013 INFO [Listener at localhost/40787] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f0046d63ce7144f5b1ee5117bed716b1 2023-03-05 19:55:57,014 DEBUG [Listener at localhost/40787] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/f5738c13f081c7a8286ce16de3ec8a21/.tmp/family1/f0046d63ce7144f5b1ee5117bed716b1 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/f5738c13f081c7a8286ce16de3ec8a21/family1/f0046d63ce7144f5b1ee5117bed716b1 2023-03-05 19:55:57,018 INFO [Listener at localhost/40787] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f0046d63ce7144f5b1ee5117bed716b1 2023-03-05 19:55:57,019 INFO [Listener at localhost/40787] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in f5738c13f081c7a8286ce16de3ec8a21/family1 of f5738c13f081c7a8286ce16de3ec8a21 into f0046d63ce7144f5b1ee5117bed716b1(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-05 19:55:57,019 DEBUG [Listener at localhost/40787] regionserver.HRegion(2289): Compaction status journal for f5738c13f081c7a8286ce16de3ec8a21: 2023-03-05 19:55:57,019 DEBUG [Listener at localhost/40787] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-05 19:55:57,020 DEBUG [Listener at localhost/40787] regionserver.HStore(1912): f5738c13f081c7a8286ce16de3ec8a21/family2 is initiating major compaction (all files) 2023-03-05 19:55:57,020 INFO [Listener at localhost/40787] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-05 19:55:57,020 INFO [Listener at localhost/40787] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-05 19:55:57,020 INFO [Listener at localhost/40787] regionserver.HRegion(2259): Starting compaction of f5738c13f081c7a8286ce16de3ec8a21/family2 in ns:testMinorCompaction,1,1678046144232.f5738c13f081c7a8286ce16de3ec8a21. 2023-03-05 19:55:57,020 INFO [Listener at localhost/40787] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/f5738c13f081c7a8286ce16de3ec8a21/family2/da7d211a0d2146d98ca748102a227893, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/f5738c13f081c7a8286ce16de3ec8a21/family2/2c49c102359e467893c28e72bfbe9318, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/f5738c13f081c7a8286ce16de3ec8a21/family2/3ad0245181f940408262b7f15b527ade, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/f5738c13f081c7a8286ce16de3ec8a21/family2/024df8f7ffcf44c4843221320fdacf20, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/f5738c13f081c7a8286ce16de3ec8a21/family2/0b51cf69b1fe409f9cac25d3afa196cc] into tmpdir=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/f5738c13f081c7a8286ce16de3ec8a21/.tmp, totalSize=25.3 K 2023-03-05 19:55:57,020 DEBUG [Listener at localhost/40787] compactions.Compactor(207): Compacting da7d211a0d2146d98ca748102a227893, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678046146346 2023-03-05 19:55:57,021 DEBUG [Listener at localhost/40787] compactions.Compactor(207): Compacting 2c49c102359e467893c28e72bfbe9318, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678046148119 2023-03-05 19:55:57,021 DEBUG [Listener at localhost/40787] compactions.Compactor(207): Compacting 3ad0245181f940408262b7f15b527ade, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678046149897 2023-03-05 19:55:57,022 DEBUG [Listener at localhost/40787] compactions.Compactor(207): Compacting 024df8f7ffcf44c4843221320fdacf20, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678046151679 2023-03-05 19:55:57,022 DEBUG [Listener at localhost/40787] compactions.Compactor(207): Compacting 0b51cf69b1fe409f9cac25d3afa196cc, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-05 19:55:57,040 INFO [Listener at localhost/40787] throttle.PressureAwareThroughputController(145): f5738c13f081c7a8286ce16de3ec8a21#family2#compaction#152 average throughput is 0.10 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-05 19:55:57,048 INFO [Listener at localhost/40787] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b2023030593876517519f47c6a6c9b75cae266152_del 2023-03-05 19:55:57,049 INFO [Listener at localhost/40787] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023030593876517519f47c6a6c9b75cae266152_del to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b2023030593876517519f47c6a6c9b75cae266152_del 2023-03-05 19:55:57,080 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/f5738c13f081c7a8286ce16de3ec8a21/family1/7ea37ba56aeb4d21a21a5d3bf889223c, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/f5738c13f081c7a8286ce16de3ec8a21/family1/5eeef2442b714333896ff4ee143861ae, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/f5738c13f081c7a8286ce16de3ec8a21/family1/0958c8dcf48e4f4e8e03df3fef051884, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/f5738c13f081c7a8286ce16de3ec8a21/family1/6efca904c2cb4ca58e88141495470acf, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/f5738c13f081c7a8286ce16de3ec8a21/family1/f867d36a504141ab970cb77201a03c1f] to archive 2023-03-05 19:55:57,080 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-05 19:55:57,082 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/f5738c13f081c7a8286ce16de3ec8a21/family1/7ea37ba56aeb4d21a21a5d3bf889223c to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/ns/testMinorCompaction/f5738c13f081c7a8286ce16de3ec8a21/family1/7ea37ba56aeb4d21a21a5d3bf889223c 2023-03-05 19:55:57,083 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/f5738c13f081c7a8286ce16de3ec8a21/family1/5eeef2442b714333896ff4ee143861ae to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/ns/testMinorCompaction/f5738c13f081c7a8286ce16de3ec8a21/family1/5eeef2442b714333896ff4ee143861ae 2023-03-05 19:55:57,083 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/f5738c13f081c7a8286ce16de3ec8a21/family1/0958c8dcf48e4f4e8e03df3fef051884 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/ns/testMinorCompaction/f5738c13f081c7a8286ce16de3ec8a21/family1/0958c8dcf48e4f4e8e03df3fef051884 2023-03-05 19:55:57,084 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/f5738c13f081c7a8286ce16de3ec8a21/family1/6efca904c2cb4ca58e88141495470acf to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/ns/testMinorCompaction/f5738c13f081c7a8286ce16de3ec8a21/family1/6efca904c2cb4ca58e88141495470acf 2023-03-05 19:55:57,085 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/f5738c13f081c7a8286ce16de3ec8a21/family1/f867d36a504141ab970cb77201a03c1f to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/ns/testMinorCompaction/f5738c13f081c7a8286ce16de3ec8a21/family1/f867d36a504141ab970cb77201a03c1f 2023-03-05 19:55:57,861 INFO [Listener at localhost/40787] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 3d846ebf795c4ef39431f41cb988f850 2023-03-05 19:55:57,862 DEBUG [Listener at localhost/40787] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/f5738c13f081c7a8286ce16de3ec8a21/.tmp/family2/3d846ebf795c4ef39431f41cb988f850 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/f5738c13f081c7a8286ce16de3ec8a21/family2/3d846ebf795c4ef39431f41cb988f850 2023-03-05 19:55:57,865 INFO [Listener at localhost/40787] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 3d846ebf795c4ef39431f41cb988f850 2023-03-05 19:55:57,866 INFO [Listener at localhost/40787] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in f5738c13f081c7a8286ce16de3ec8a21/family2 of f5738c13f081c7a8286ce16de3ec8a21 into 3d846ebf795c4ef39431f41cb988f850(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-05 19:55:57,866 DEBUG [Listener at localhost/40787] regionserver.HRegion(2289): Compaction status journal for f5738c13f081c7a8286ce16de3ec8a21: 2023-03-05 19:55:57,866 DEBUG [Listener at localhost/40787] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-05 19:55:57,866 DEBUG [Listener at localhost/40787] regionserver.HStore(1912): 6fdfc14e8c700cdbd0a252655cbff4b4/family1 is initiating major compaction (all files) 2023-03-05 19:55:57,866 INFO [Listener at localhost/40787] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-05 19:55:57,866 INFO [Listener at localhost/40787] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-05 19:55:57,867 INFO [Listener at localhost/40787] regionserver.HRegion(2259): Starting compaction of 6fdfc14e8c700cdbd0a252655cbff4b4/family1 in ns:testMinorCompaction,,1678046144232.6fdfc14e8c700cdbd0a252655cbff4b4. 2023-03-05 19:55:57,867 INFO [Listener at localhost/40787] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/6fdfc14e8c700cdbd0a252655cbff4b4/family1/5032d1ccb70d49d6a55d3e09c1b22529, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/6fdfc14e8c700cdbd0a252655cbff4b4/family1/099d290778644902a97fce25291e5726, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/6fdfc14e8c700cdbd0a252655cbff4b4/family1/dc01625027384c25a034d7c00743e9ba, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/6fdfc14e8c700cdbd0a252655cbff4b4/family1/5178bc46025e4fa4ae8453540ec38ce2, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/6fdfc14e8c700cdbd0a252655cbff4b4/family1/629486c6ac1f4c33be772dfa9f232be7] into tmpdir=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/6fdfc14e8c700cdbd0a252655cbff4b4/.tmp, totalSize=26.6 K 2023-03-05 19:55:57,867 DEBUG [Listener at localhost/40787] compactions.Compactor(207): Compacting 5032d1ccb70d49d6a55d3e09c1b22529, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678046146346 2023-03-05 19:55:57,867 DEBUG [Listener at localhost/40787] compactions.Compactor(207): Compacting 099d290778644902a97fce25291e5726, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678046148119 2023-03-05 19:55:57,868 DEBUG [Listener at localhost/40787] compactions.Compactor(207): Compacting dc01625027384c25a034d7c00743e9ba, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678046149897 2023-03-05 19:55:57,868 DEBUG [Listener at localhost/40787] compactions.Compactor(207): Compacting 5178bc46025e4fa4ae8453540ec38ce2, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678046151679 2023-03-05 19:55:57,868 DEBUG [Listener at localhost/40787] compactions.Compactor(207): Compacting 629486c6ac1f4c33be772dfa9f232be7, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-05 19:55:57,876 INFO [Listener at localhost/40787] throttle.PressureAwareThroughputController(145): 6fdfc14e8c700cdbd0a252655cbff4b4#family1#compaction#153 average throughput is 0.83 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-05 19:55:57,880 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/f5738c13f081c7a8286ce16de3ec8a21/family2/da7d211a0d2146d98ca748102a227893, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/f5738c13f081c7a8286ce16de3ec8a21/family2/2c49c102359e467893c28e72bfbe9318, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/f5738c13f081c7a8286ce16de3ec8a21/family2/3ad0245181f940408262b7f15b527ade, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/f5738c13f081c7a8286ce16de3ec8a21/family2/024df8f7ffcf44c4843221320fdacf20, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/f5738c13f081c7a8286ce16de3ec8a21/family2/0b51cf69b1fe409f9cac25d3afa196cc] to archive 2023-03-05 19:55:57,880 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-05 19:55:57,882 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/f5738c13f081c7a8286ce16de3ec8a21/family2/da7d211a0d2146d98ca748102a227893 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/ns/testMinorCompaction/f5738c13f081c7a8286ce16de3ec8a21/family2/da7d211a0d2146d98ca748102a227893 2023-03-05 19:55:57,882 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/f5738c13f081c7a8286ce16de3ec8a21/family2/2c49c102359e467893c28e72bfbe9318 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/ns/testMinorCompaction/f5738c13f081c7a8286ce16de3ec8a21/family2/2c49c102359e467893c28e72bfbe9318 2023-03-05 19:55:57,883 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/f5738c13f081c7a8286ce16de3ec8a21/family2/3ad0245181f940408262b7f15b527ade to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/ns/testMinorCompaction/f5738c13f081c7a8286ce16de3ec8a21/family2/3ad0245181f940408262b7f15b527ade 2023-03-05 19:55:57,884 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/f5738c13f081c7a8286ce16de3ec8a21/family2/024df8f7ffcf44c4843221320fdacf20 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/ns/testMinorCompaction/f5738c13f081c7a8286ce16de3ec8a21/family2/024df8f7ffcf44c4843221320fdacf20 2023-03-05 19:55:57,885 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/f5738c13f081c7a8286ce16de3ec8a21/family2/0b51cf69b1fe409f9cac25d3afa196cc to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/ns/testMinorCompaction/f5738c13f081c7a8286ce16de3ec8a21/family2/0b51cf69b1fe409f9cac25d3afa196cc 2023-03-05 19:55:58,284 INFO [Listener at localhost/40787] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e202303052ad3588bf4d6400aaa6cd3d0dae659a4_del 2023-03-05 19:55:58,284 INFO [Listener at localhost/40787] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303052ad3588bf4d6400aaa6cd3d0dae659a4_del to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e202303052ad3588bf4d6400aaa6cd3d0dae659a4_del 2023-03-05 19:55:59,098 INFO [Listener at localhost/40787] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d5cd4b596a6741a2b35424cfa920f122 2023-03-05 19:55:59,099 DEBUG [Listener at localhost/40787] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/6fdfc14e8c700cdbd0a252655cbff4b4/.tmp/family1/d5cd4b596a6741a2b35424cfa920f122 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/6fdfc14e8c700cdbd0a252655cbff4b4/family1/d5cd4b596a6741a2b35424cfa920f122 2023-03-05 19:55:59,103 INFO [Listener at localhost/40787] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d5cd4b596a6741a2b35424cfa920f122 2023-03-05 19:55:59,104 INFO [Listener at localhost/40787] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 6fdfc14e8c700cdbd0a252655cbff4b4/family1 of 6fdfc14e8c700cdbd0a252655cbff4b4 into d5cd4b596a6741a2b35424cfa920f122(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-05 19:55:59,104 DEBUG [Listener at localhost/40787] regionserver.HRegion(2289): Compaction status journal for 6fdfc14e8c700cdbd0a252655cbff4b4: 2023-03-05 19:55:59,104 DEBUG [Listener at localhost/40787] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-05 19:55:59,104 DEBUG [Listener at localhost/40787] regionserver.HStore(1912): 6fdfc14e8c700cdbd0a252655cbff4b4/family2 is initiating major compaction (all files) 2023-03-05 19:55:59,104 INFO [Listener at localhost/40787] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-05 19:55:59,104 INFO [Listener at localhost/40787] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-05 19:55:59,104 INFO [Listener at localhost/40787] regionserver.HRegion(2259): Starting compaction of 6fdfc14e8c700cdbd0a252655cbff4b4/family2 in ns:testMinorCompaction,,1678046144232.6fdfc14e8c700cdbd0a252655cbff4b4. 2023-03-05 19:55:59,104 INFO [Listener at localhost/40787] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/6fdfc14e8c700cdbd0a252655cbff4b4/family2/7bedf8a039ea4580bbf505c8d84bd02f, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/6fdfc14e8c700cdbd0a252655cbff4b4/family2/df0f37123f2d4c53988bfc6eef9a8b92, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/6fdfc14e8c700cdbd0a252655cbff4b4/family2/90daa54ea7c74225bafe739c1089cfe4, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/6fdfc14e8c700cdbd0a252655cbff4b4/family2/f43387a93a0e4a619d881dc70625ee18, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/6fdfc14e8c700cdbd0a252655cbff4b4/family2/d0040a46052e415597bf954de7cb6c4c] into tmpdir=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/6fdfc14e8c700cdbd0a252655cbff4b4/.tmp, totalSize=25.3 K 2023-03-05 19:55:59,105 DEBUG [Listener at localhost/40787] compactions.Compactor(207): Compacting 7bedf8a039ea4580bbf505c8d84bd02f, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678046146346 2023-03-05 19:55:59,105 DEBUG [Listener at localhost/40787] compactions.Compactor(207): Compacting df0f37123f2d4c53988bfc6eef9a8b92, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678046148119 2023-03-05 19:55:59,105 DEBUG [Listener at localhost/40787] compactions.Compactor(207): Compacting 90daa54ea7c74225bafe739c1089cfe4, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678046149897 2023-03-05 19:55:59,105 DEBUG [Listener at localhost/40787] compactions.Compactor(207): Compacting f43387a93a0e4a619d881dc70625ee18, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678046151679 2023-03-05 19:55:59,106 DEBUG [Listener at localhost/40787] compactions.Compactor(207): Compacting d0040a46052e415597bf954de7cb6c4c, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-05 19:55:59,124 INFO [Listener at localhost/40787] throttle.PressureAwareThroughputController(145): 6fdfc14e8c700cdbd0a252655cbff4b4#family2#compaction#154 average throughput is 0.08 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-05 19:55:59,185 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/6fdfc14e8c700cdbd0a252655cbff4b4/family1/5032d1ccb70d49d6a55d3e09c1b22529, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/6fdfc14e8c700cdbd0a252655cbff4b4/family1/099d290778644902a97fce25291e5726, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/6fdfc14e8c700cdbd0a252655cbff4b4/family1/dc01625027384c25a034d7c00743e9ba, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/6fdfc14e8c700cdbd0a252655cbff4b4/family1/5178bc46025e4fa4ae8453540ec38ce2, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/6fdfc14e8c700cdbd0a252655cbff4b4/family1/629486c6ac1f4c33be772dfa9f232be7] to archive 2023-03-05 19:55:59,186 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-05 19:55:59,188 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/6fdfc14e8c700cdbd0a252655cbff4b4/family1/5032d1ccb70d49d6a55d3e09c1b22529 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/ns/testMinorCompaction/6fdfc14e8c700cdbd0a252655cbff4b4/family1/5032d1ccb70d49d6a55d3e09c1b22529 2023-03-05 19:55:59,189 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/6fdfc14e8c700cdbd0a252655cbff4b4/family1/099d290778644902a97fce25291e5726 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/ns/testMinorCompaction/6fdfc14e8c700cdbd0a252655cbff4b4/family1/099d290778644902a97fce25291e5726 2023-03-05 19:55:59,190 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/6fdfc14e8c700cdbd0a252655cbff4b4/family1/dc01625027384c25a034d7c00743e9ba to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/ns/testMinorCompaction/6fdfc14e8c700cdbd0a252655cbff4b4/family1/dc01625027384c25a034d7c00743e9ba 2023-03-05 19:55:59,191 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/6fdfc14e8c700cdbd0a252655cbff4b4/family1/5178bc46025e4fa4ae8453540ec38ce2 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/ns/testMinorCompaction/6fdfc14e8c700cdbd0a252655cbff4b4/family1/5178bc46025e4fa4ae8453540ec38ce2 2023-03-05 19:55:59,192 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/6fdfc14e8c700cdbd0a252655cbff4b4/family1/629486c6ac1f4c33be772dfa9f232be7 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/ns/testMinorCompaction/6fdfc14e8c700cdbd0a252655cbff4b4/family1/629486c6ac1f4c33be772dfa9f232be7 2023-03-05 19:55:59,531 INFO [Listener at localhost/40787] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e2023030515f227fd7953459282bd41046134da9a_del 2023-03-05 19:55:59,531 INFO [Listener at localhost/40787] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023030515f227fd7953459282bd41046134da9a_del to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e2023030515f227fd7953459282bd41046134da9a_del 2023-03-05 19:56:00,344 INFO [Listener at localhost/40787] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 47f7d5ff59854004b1b14a293afd25fc 2023-03-05 19:56:00,345 DEBUG [Listener at localhost/40787] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/6fdfc14e8c700cdbd0a252655cbff4b4/.tmp/family2/47f7d5ff59854004b1b14a293afd25fc as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/6fdfc14e8c700cdbd0a252655cbff4b4/family2/47f7d5ff59854004b1b14a293afd25fc 2023-03-05 19:56:00,348 INFO [Listener at localhost/40787] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 47f7d5ff59854004b1b14a293afd25fc 2023-03-05 19:56:00,349 INFO [Listener at localhost/40787] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 6fdfc14e8c700cdbd0a252655cbff4b4/family2 of 6fdfc14e8c700cdbd0a252655cbff4b4 into 47f7d5ff59854004b1b14a293afd25fc(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-05 19:56:00,349 DEBUG [Listener at localhost/40787] regionserver.HRegion(2289): Compaction status journal for 6fdfc14e8c700cdbd0a252655cbff4b4: 2023-03-05 19:56:00,354 DEBUG [Listener at localhost/40787] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-05 19:56:00,357 DEBUG [Listener at localhost/40787] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-05 19:56:00,362 DEBUG [Listener at localhost/40787] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-05 19:56:00,363 DEBUG [Listener at localhost/40787] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-05 19:56:00,371 INFO [Listener at localhost/40787] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-03-05 19:56:00,381 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/6fdfc14e8c700cdbd0a252655cbff4b4/family2/7bedf8a039ea4580bbf505c8d84bd02f, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/6fdfc14e8c700cdbd0a252655cbff4b4/family2/df0f37123f2d4c53988bfc6eef9a8b92, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/6fdfc14e8c700cdbd0a252655cbff4b4/family2/90daa54ea7c74225bafe739c1089cfe4, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/6fdfc14e8c700cdbd0a252655cbff4b4/family2/f43387a93a0e4a619d881dc70625ee18, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/6fdfc14e8c700cdbd0a252655cbff4b4/family2/d0040a46052e415597bf954de7cb6c4c] to archive 2023-03-05 19:56:00,384 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-05 19:56:00,386 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/6fdfc14e8c700cdbd0a252655cbff4b4/family2/7bedf8a039ea4580bbf505c8d84bd02f to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/ns/testMinorCompaction/6fdfc14e8c700cdbd0a252655cbff4b4/family2/7bedf8a039ea4580bbf505c8d84bd02f 2023-03-05 19:56:00,387 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/6fdfc14e8c700cdbd0a252655cbff4b4/family2/df0f37123f2d4c53988bfc6eef9a8b92 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/ns/testMinorCompaction/6fdfc14e8c700cdbd0a252655cbff4b4/family2/df0f37123f2d4c53988bfc6eef9a8b92 2023-03-05 19:56:00,389 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/6fdfc14e8c700cdbd0a252655cbff4b4/family2/90daa54ea7c74225bafe739c1089cfe4 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/ns/testMinorCompaction/6fdfc14e8c700cdbd0a252655cbff4b4/family2/90daa54ea7c74225bafe739c1089cfe4 2023-03-05 19:56:00,390 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/6fdfc14e8c700cdbd0a252655cbff4b4/family2/f43387a93a0e4a619d881dc70625ee18 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/ns/testMinorCompaction/6fdfc14e8c700cdbd0a252655cbff4b4/family2/f43387a93a0e4a619d881dc70625ee18 2023-03-05 19:56:00,392 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/6fdfc14e8c700cdbd0a252655cbff4b4/family2/d0040a46052e415597bf954de7cb6c4c to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/ns/testMinorCompaction/6fdfc14e8c700cdbd0a252655cbff4b4/family2/d0040a46052e415597bf954de7cb6c4c 2023-03-05 19:56:00,400 INFO [Listener at localhost/40787] compactions.PartitionedMobCompactor(306): The compaction type is PART_FILES, the request has 3 del files, 3 selected files, and 0 irrelevant files table 'ns:testMinorCompaction' and column 'family1' 2023-03-05 19:56:00,405 INFO [Listener at localhost/40787] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e202303052ad3588bf4d6400aaa6cd3d0dae659a4_del 2023-03-05 19:56:00,412 INFO [Listener at localhost/40787] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b202303051654ba92a3414768a48ff37a27675159_del 2023-03-05 19:56:00,418 INFO [Listener at localhost/40787] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230305e61d8371ebb94064a8df627c46b77a99_del 2023-03-05 19:56:00,418 INFO [Listener at localhost/40787] compactions.PartitionedMobCompactor(350): After merging, there are 3 del files. table='ns:testMinorCompaction' column='family1' 2023-03-05 19:56:00,418 DEBUG [Listener at localhost/40787] zookeeper.ReadOnlyZKClient(139): Connect 0x3894c415 to 127.0.0.1:62957 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-05 19:56:00,441 DEBUG [Listener at localhost/40787] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@57b6b55c, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-05 19:56:00,442 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230305 for table='ns:testMinorCompaction' and column='family1' 2023-03-05 19:56:00,443 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230305 for table='ns:testMinorCompaction' and column='family1' 2023-03-05 19:56:00,442 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c4ca4238a0b923820dcc509a6f75849b20230305 for table='ns:testMinorCompaction' and column='family1' 2023-03-05 19:56:00,885 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230305e9609633779a44e7af4090c8b55c92c9 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b20230305e9609633779a44e7af4090c8b55c92c9 2023-03-05 19:56:00,886 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230305 table='ns:testMinorCompaction' column='family1' 2023-03-05 19:56:00,887 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230305235fb172d01347ee9edca321294c9c5b to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c20230305235fb172d01347ee9edca321294c9c5b 2023-03-05 19:56:00,888 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230305 table='ns:testMinorCompaction' column='family1' 2023-03-05 19:56:00,898 DEBUG [MobFileCompactionChore-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-05 19:56:00,902 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:33746, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-05 19:56:00,907 DEBUG [MobFileCompactionChore-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-05 19:56:00,910 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:50784, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-05 19:56:00,921 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230305/family1/b42b24bd786745b0a60ead90e788a90a first=Optional[2\x00\x00\x00\x01] last=Optional[2\x00\x00\x00\x01] 2023-03-05 19:56:00,922 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=ns:testMinorCompaction,2,1678046144232.25ed4806e34b655e8718967079efb560., hostname=jenkins-hbase4.apache.org,37037,1678046018983, seqNum=2 for row 2 with hfile group [{family1,hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230305/family1/b42b24bd786745b0a60ead90e788a90a}] 2023-03-05 19:56:00,922 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230305/family1/65ddb0111b4d417cb2ea3e07946b035c first=Optional[1\x00\x00\x00\x01] last=Optional[1\x00\x00\x00\x01] 2023-03-05 19:56:00,922 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=ns:testMinorCompaction,1,1678046144232.f5738c13f081c7a8286ce16de3ec8a21., hostname=jenkins-hbase4.apache.org,37037,1678046018983, seqNum=2 for row 1 with hfile group [{family1,hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230305/family1/65ddb0111b4d417cb2ea3e07946b035c}] 2023-03-05 19:56:00,929 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.HStore(763): Validating hfile at hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230305/family1/65ddb0111b4d417cb2ea3e07946b035c for inclusion in f5738c13f081c7a8286ce16de3ec8a21/family1 2023-03-05 19:56:00,930 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37037] regionserver.HStore(763): Validating hfile at hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230305/family1/b42b24bd786745b0a60ead90e788a90a for inclusion in 25ed4806e34b655e8718967079efb560/family1 2023-03-05 19:56:00,932 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.HStore(775): HFile bounds: first=1\x00\x00\x00\x01 last=1\x00\x00\x00\x01 2023-03-05 19:56:00,932 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.HStore(777): Region bounds: first=1 last=2 2023-03-05 19:56:00,932 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37037] regionserver.HStore(775): HFile bounds: first=2\x00\x00\x00\x01 last=2\x00\x00\x00\x01 2023-03-05 19:56:00,932 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.HRegion(2446): Flush status journal for f5738c13f081c7a8286ce16de3ec8a21: 2023-03-05 19:56:00,932 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37037] regionserver.HStore(777): Region bounds: first=2 last= 2023-03-05 19:56:00,932 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37037] regionserver.HRegion(2446): Flush status journal for 25ed4806e34b655e8718967079efb560: 2023-03-05 19:56:00,933 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230305/family1/65ddb0111b4d417cb2ea3e07946b035c to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__ns_testMinorCompaction__efuekunn85a690a35fu7g7p8e76ufcstn6d6nlen2a1u10pk8mr78rc7glip24s4/family1/65ddb0111b4d417cb2ea3e07946b035c 2023-03-05 19:56:00,933 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37037] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230305/family1/b42b24bd786745b0a60ead90e788a90a to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__ns_testMinorCompaction__s3lcqc63hnb58n517jpu390s43jjf9k9alua6p53eoln1rvpi5b6drhrcfka9h8j/family1/b42b24bd786745b0a60ead90e788a90a 2023-03-05 19:56:00,935 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37037] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__ns_testMinorCompaction__s3lcqc63hnb58n517jpu390s43jjf9k9alua6p53eoln1rvpi5b6drhrcfka9h8j/family1/b42b24bd786745b0a60ead90e788a90a as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/25ed4806e34b655e8718967079efb560/family1/88622605928445e9a4d31d4b7cee548d_SeqId_28_ 2023-03-05 19:56:00,936 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37037] regionserver.HStore(849): Loaded HFile hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__ns_testMinorCompaction__s3lcqc63hnb58n517jpu390s43jjf9k9alua6p53eoln1rvpi5b6drhrcfka9h8j/family1/b42b24bd786745b0a60ead90e788a90a into 25ed4806e34b655e8718967079efb560/family1 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/25ed4806e34b655e8718967079efb560/family1/88622605928445e9a4d31d4b7cee548d_SeqId_28_ - updating store file list. 2023-03-05 19:56:00,937 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__ns_testMinorCompaction__efuekunn85a690a35fu7g7p8e76ufcstn6d6nlen2a1u10pk8mr78rc7glip24s4/family1/65ddb0111b4d417cb2ea3e07946b035c as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/f5738c13f081c7a8286ce16de3ec8a21/family1/0959945a8d42416e926c4ca042a6426e_SeqId_28_ 2023-03-05 19:56:00,938 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.HStore(849): Loaded HFile hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__ns_testMinorCompaction__efuekunn85a690a35fu7g7p8e76ufcstn6d6nlen2a1u10pk8mr78rc7glip24s4/family1/65ddb0111b4d417cb2ea3e07946b035c into f5738c13f081c7a8286ce16de3ec8a21/family1 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/f5738c13f081c7a8286ce16de3ec8a21/family1/0959945a8d42416e926c4ca042a6426e_SeqId_28_ - updating store file list. 2023-03-05 19:56:00,940 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37037] regionserver.HStore(882): Loaded HFile hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/25ed4806e34b655e8718967079efb560/family1/88622605928445e9a4d31d4b7cee548d_SeqId_28_ into 25ed4806e34b655e8718967079efb560/family1 2023-03-05 19:56:00,940 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37037] regionserver.HStore(855): Successfully loaded hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__ns_testMinorCompaction__s3lcqc63hnb58n517jpu390s43jjf9k9alua6p53eoln1rvpi5b6drhrcfka9h8j/family1/b42b24bd786745b0a60ead90e788a90a into 25ed4806e34b655e8718967079efb560/family1 (new location: hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/25ed4806e34b655e8718967079efb560/family1/88622605928445e9a4d31d4b7cee548d_SeqId_28_) 2023-03-05 19:56:00,940 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37037] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__ns_testMinorCompaction__s3lcqc63hnb58n517jpu390s43jjf9k9alua6p53eoln1rvpi5b6drhrcfka9h8j/family1/b42b24bd786745b0a60ead90e788a90a 2023-03-05 19:56:00,941 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.HStore(882): Loaded HFile hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/f5738c13f081c7a8286ce16de3ec8a21/family1/0959945a8d42416e926c4ca042a6426e_SeqId_28_ into f5738c13f081c7a8286ce16de3ec8a21/family1 2023-03-05 19:56:00,941 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.HStore(855): Successfully loaded hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__ns_testMinorCompaction__efuekunn85a690a35fu7g7p8e76ufcstn6d6nlen2a1u10pk8mr78rc7glip24s4/family1/65ddb0111b4d417cb2ea3e07946b035c into f5738c13f081c7a8286ce16de3ec8a21/family1 (new location: hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/f5738c13f081c7a8286ce16de3ec8a21/family1/0959945a8d42416e926c4ca042a6426e_SeqId_28_) 2023-03-05 19:56:00,942 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__ns_testMinorCompaction__efuekunn85a690a35fu7g7p8e76ufcstn6d6nlen2a1u10pk8mr78rc7glip24s4/family1/65ddb0111b4d417cb2ea3e07946b035c 2023-03-05 19:56:00,943 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230305 table='ns:testMinorCompaction' column='family1' 2023-03-05 19:56:00,944 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(360): Archiving compacted files. 2023-03-05 19:56:00,944 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230305 table='ns:testMinorCompaction' column='family1' 2023-03-05 19:56:00,944 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-05 19:56:00,945 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c20230305b514ad0f1fb24aba8224f018265df055 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c20230305b514ad0f1fb24aba8224f018265df055 2023-03-05 19:56:00,945 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(508): Table ns:testMinorCompaction 1 expired mob files are deleted 2023-03-05 19:56:00,945 DEBUG [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230305' 2023-03-05 19:56:00,946 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b202303055525fb319dd04a3ba26b40c244794c22 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b202303055525fb319dd04a3ba26b40c244794c22 2023-03-05 19:56:00,946 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(508): Table ns:testMinorCompaction 1 expired mob files are deleted 2023-03-05 19:56:00,946 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230305 for table='ns:testMinorCompaction' and column='family1' 2023-03-05 19:56:00,946 DEBUG [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230305' 2023-03-05 19:56:00,946 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=c4ca4238a0b923820dcc509a6f75849b20230305 for table='ns:testMinorCompaction' and column='family1' 2023-03-05 19:56:01,287 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230305731648200f2941fea91831a0d51b9704 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e20230305731648200f2941fea91831a0d51b9704 2023-03-05 19:56:01,288 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230305 table='ns:testMinorCompaction' column='family1' 2023-03-05 19:56:01,311 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230305/family1/f00d83427c354292aa558fba32d557ae first=Optional[0\x00\x00\x00\x01] last=Optional[0\x00\x00\x00\x01] 2023-03-05 19:56:01,312 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=ns:testMinorCompaction,,1678046144232.6fdfc14e8c700cdbd0a252655cbff4b4., hostname=jenkins-hbase4.apache.org,37037,1678046018983, seqNum=2 for row with hfile group [{family1,hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230305/family1/f00d83427c354292aa558fba32d557ae}] 2023-03-05 19:56:01,314 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.HStore(763): Validating hfile at hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230305/family1/f00d83427c354292aa558fba32d557ae for inclusion in 6fdfc14e8c700cdbd0a252655cbff4b4/family1 2023-03-05 19:56:01,316 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.HStore(775): HFile bounds: first=0\x00\x00\x00\x01 last=0\x00\x00\x00\x01 2023-03-05 19:56:01,316 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.HStore(777): Region bounds: first= last=1 2023-03-05 19:56:01,316 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.HRegion(2446): Flush status journal for 6fdfc14e8c700cdbd0a252655cbff4b4: 2023-03-05 19:56:01,317 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230305/family1/f00d83427c354292aa558fba32d557ae to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__ns_testMinorCompaction__95qb9sm5fkc4glmedoheivf4nln0t35c4p2peohks7bsvrb5icmdk5ovr2l99qg5/family1/f00d83427c354292aa558fba32d557ae 2023-03-05 19:56:01,318 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__ns_testMinorCompaction__95qb9sm5fkc4glmedoheivf4nln0t35c4p2peohks7bsvrb5icmdk5ovr2l99qg5/family1/f00d83427c354292aa558fba32d557ae as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/6fdfc14e8c700cdbd0a252655cbff4b4/family1/78d617ab33f94e34923699a1140526bd_SeqId_28_ 2023-03-05 19:56:01,319 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.HStore(849): Loaded HFile hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__ns_testMinorCompaction__95qb9sm5fkc4glmedoheivf4nln0t35c4p2peohks7bsvrb5icmdk5ovr2l99qg5/family1/f00d83427c354292aa558fba32d557ae into 6fdfc14e8c700cdbd0a252655cbff4b4/family1 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/6fdfc14e8c700cdbd0a252655cbff4b4/family1/78d617ab33f94e34923699a1140526bd_SeqId_28_ - updating store file list. 2023-03-05 19:56:01,322 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.HStore(882): Loaded HFile hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/6fdfc14e8c700cdbd0a252655cbff4b4/family1/78d617ab33f94e34923699a1140526bd_SeqId_28_ into 6fdfc14e8c700cdbd0a252655cbff4b4/family1 2023-03-05 19:56:01,322 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.HStore(855): Successfully loaded hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__ns_testMinorCompaction__95qb9sm5fkc4glmedoheivf4nln0t35c4p2peohks7bsvrb5icmdk5ovr2l99qg5/family1/f00d83427c354292aa558fba32d557ae into 6fdfc14e8c700cdbd0a252655cbff4b4/family1 (new location: hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/6fdfc14e8c700cdbd0a252655cbff4b4/family1/78d617ab33f94e34923699a1140526bd_SeqId_28_) 2023-03-05 19:56:01,322 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__ns_testMinorCompaction__95qb9sm5fkc4glmedoheivf4nln0t35c4p2peohks7bsvrb5icmdk5ovr2l99qg5/family1/f00d83427c354292aa558fba32d557ae 2023-03-05 19:56:01,324 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230305 table='ns:testMinorCompaction' column='family1' 2023-03-05 19:56:01,324 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(360): Archiving compacted files. 2023-03-05 19:56:01,325 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e202303057ca0feb1e1784191832759b3dc4ad143 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e202303057ca0feb1e1784191832759b3dc4ad143 2023-03-05 19:56:01,325 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(508): Table ns:testMinorCompaction 1 expired mob files are deleted 2023-03-05 19:56:01,326 DEBUG [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230305' 2023-03-05 19:56:01,326 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230305 for table='ns:testMinorCompaction' and column='family1' 2023-03-05 19:56:01,326 INFO [Listener at localhost/40787] compactions.PartitionedMobCompactor(354): After compaction, there are 3 mob files. table='ns:testMinorCompaction' column='family1' 2023-03-05 19:56:01,328 DEBUG [Listener at localhost/40787] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-05 19:56:01,330 DEBUG [Listener at localhost/40787] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-05 19:56:01,332 DEBUG [Listener at localhost/40787] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-05 19:56:01,333 DEBUG [Listener at localhost/40787] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-05 19:56:01,353 INFO [Listener at localhost/40787] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMinorCompaction Thread=253 (was 289), OpenFileDescriptor=639 (was 760), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=86 (was 90), ProcessCount=171 (was 171), AvailableMemoryMB=4503 (was 4573) 2023-03-05 19:56:01,365 INFO [Listener at localhost/40787] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMinorCompactionWithMonthlyPolicy Thread=253, OpenFileDescriptor=639, MaxFileDescriptor=60000, SystemLoadAverage=86, ProcessCount=171, AvailableMemoryMB=4502 2023-03-05 19:56:01,366 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMinorCompactionWithMonthlyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-03-05 19:56:01,367 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure2.ProcedureExecutor(1029): Stored pid=112, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:01,368 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_PRE_OPERATION 2023-03-05 19:56:01,368 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMinorCompactionWithMonthlyPolicy" procId is: 112 2023-03-05 19:56:01,369 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-05 19:56:01,369 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-03-05 19:56:01,370 DEBUG [HFileArchiver-19] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/.tmp/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258 2023-03-05 19:56:01,370 DEBUG [HFileArchiver-19] backup.HFileArchiver(153): Directory hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/.tmp/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258 empty. 2023-03-05 19:56:01,371 DEBUG [HFileArchiver-19] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/.tmp/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258 2023-03-05 19:56:01,371 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived testMinorCompactionWithMonthlyPolicy regions 2023-03-05 19:56:01,470 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-03-05 19:56:01,671 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-03-05 19:56:01,777 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/.tmp/data/default/testMinorCompactionWithMonthlyPolicy/.tabledesc/.tableinfo.0000000001 2023-03-05 19:56:01,779 INFO [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(7675): creating {ENCODED => 014dc240b491a64066ab7b01d2b21258, NAME => 'testMinorCompactionWithMonthlyPolicy,,1678046161366.014dc240b491a64066ab7b01d2b21258.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMinorCompactionWithMonthlyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}, regionDir=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/.tmp 2023-03-05 19:56:01,972 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-03-05 19:56:02,184 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(866): Instantiated testMinorCompactionWithMonthlyPolicy,,1678046161366.014dc240b491a64066ab7b01d2b21258.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-05 19:56:02,184 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1604): Closing 014dc240b491a64066ab7b01d2b21258, disabling compactions & flushes 2023-03-05 19:56:02,184 INFO [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1626): Closing region testMinorCompactionWithMonthlyPolicy,,1678046161366.014dc240b491a64066ab7b01d2b21258. 2023-03-05 19:56:02,184 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMinorCompactionWithMonthlyPolicy,,1678046161366.014dc240b491a64066ab7b01d2b21258. 2023-03-05 19:56:02,184 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1714): Acquired close lock on testMinorCompactionWithMonthlyPolicy,,1678046161366.014dc240b491a64066ab7b01d2b21258. after waiting 0 ms 2023-03-05 19:56:02,184 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1724): Updates disabled for region testMinorCompactionWithMonthlyPolicy,,1678046161366.014dc240b491a64066ab7b01d2b21258. 2023-03-05 19:56:02,184 INFO [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1838): Closed testMinorCompactionWithMonthlyPolicy,,1678046161366.014dc240b491a64066ab7b01d2b21258. 2023-03-05 19:56:02,184 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1558): Region close journal for 014dc240b491a64066ab7b01d2b21258: 2023-03-05 19:56:02,186 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_ADD_TO_META 2023-03-05 19:56:02,187 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMinorCompactionWithMonthlyPolicy,,1678046161366.014dc240b491a64066ab7b01d2b21258.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1678046162187"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678046162187"}]},"ts":"1678046162187"} 2023-03-05 19:56:02,188 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-03-05 19:56:02,189 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-05 19:56:02,189 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMinorCompactionWithMonthlyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678046162189"}]},"ts":"1678046162189"} 2023-03-05 19:56:02,190 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testMinorCompactionWithMonthlyPolicy, state=ENABLING in hbase:meta 2023-03-05 19:56:02,193 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMinorCompactionWithMonthlyPolicy, region=014dc240b491a64066ab7b01d2b21258, ASSIGN}] 2023-03-05 19:56:02,195 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMinorCompactionWithMonthlyPolicy, region=014dc240b491a64066ab7b01d2b21258, ASSIGN 2023-03-05 19:56:02,195 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMinorCompactionWithMonthlyPolicy, region=014dc240b491a64066ab7b01d2b21258, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,37037,1678046018983; forceNewPlan=false, retain=false 2023-03-05 19:56:02,346 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=014dc240b491a64066ab7b01d2b21258, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:02,346 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMinorCompactionWithMonthlyPolicy,,1678046161366.014dc240b491a64066ab7b01d2b21258.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1678046162346"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678046162346"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678046162346"}]},"ts":"1678046162346"} 2023-03-05 19:56:02,348 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=114, ppid=113, state=RUNNABLE; OpenRegionProcedure 014dc240b491a64066ab7b01d2b21258, server=jenkins-hbase4.apache.org,37037,1678046018983}] 2023-03-05 19:56:02,473 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-03-05 19:56:02,502 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMinorCompactionWithMonthlyPolicy,,1678046161366.014dc240b491a64066ab7b01d2b21258. 2023-03-05 19:56:02,502 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 014dc240b491a64066ab7b01d2b21258, NAME => 'testMinorCompactionWithMonthlyPolicy,,1678046161366.014dc240b491a64066ab7b01d2b21258.', STARTKEY => '', ENDKEY => ''} 2023-03-05 19:56:02,502 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompactionWithMonthlyPolicy 014dc240b491a64066ab7b01d2b21258 2023-03-05 19:56:02,502 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMinorCompactionWithMonthlyPolicy,,1678046161366.014dc240b491a64066ab7b01d2b21258.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-05 19:56:02,502 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 014dc240b491a64066ab7b01d2b21258 2023-03-05 19:56:02,502 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 014dc240b491a64066ab7b01d2b21258 2023-03-05 19:56:02,503 INFO [StoreOpener-014dc240b491a64066ab7b01d2b21258-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 014dc240b491a64066ab7b01d2b21258 2023-03-05 19:56:02,504 DEBUG [StoreOpener-014dc240b491a64066ab7b01d2b21258-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258/family1 2023-03-05 19:56:02,504 DEBUG [StoreOpener-014dc240b491a64066ab7b01d2b21258-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258/family1 2023-03-05 19:56:02,505 INFO [StoreOpener-014dc240b491a64066ab7b01d2b21258-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 014dc240b491a64066ab7b01d2b21258 columnFamilyName family1 2023-03-05 19:56:02,506 INFO [StoreOpener-014dc240b491a64066ab7b01d2b21258-1] regionserver.HStore(310): Store=014dc240b491a64066ab7b01d2b21258/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-05 19:56:02,507 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258 2023-03-05 19:56:02,507 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258 2023-03-05 19:56:02,509 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 014dc240b491a64066ab7b01d2b21258 2023-03-05 19:56:02,511 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-05 19:56:02,511 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 014dc240b491a64066ab7b01d2b21258; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59389697, jitterRate=-0.11502455174922943}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-05 19:56:02,511 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 014dc240b491a64066ab7b01d2b21258: 2023-03-05 19:56:02,512 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMinorCompactionWithMonthlyPolicy,,1678046161366.014dc240b491a64066ab7b01d2b21258., pid=114, masterSystemTime=1678046162499 2023-03-05 19:56:02,513 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMinorCompactionWithMonthlyPolicy,,1678046161366.014dc240b491a64066ab7b01d2b21258. 2023-03-05 19:56:02,513 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMinorCompactionWithMonthlyPolicy,,1678046161366.014dc240b491a64066ab7b01d2b21258. 2023-03-05 19:56:02,513 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=014dc240b491a64066ab7b01d2b21258, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:02,514 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMinorCompactionWithMonthlyPolicy,,1678046161366.014dc240b491a64066ab7b01d2b21258.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1678046162513"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678046162513"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678046162513"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678046162513"}]},"ts":"1678046162513"} 2023-03-05 19:56:02,515 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=114, resume processing ppid=113 2023-03-05 19:56:02,516 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=114, ppid=113, state=SUCCESS; OpenRegionProcedure 014dc240b491a64066ab7b01d2b21258, server=jenkins-hbase4.apache.org,37037,1678046018983 in 167 msec 2023-03-05 19:56:02,517 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=113, resume processing ppid=112 2023-03-05 19:56:02,517 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testMinorCompactionWithMonthlyPolicy, region=014dc240b491a64066ab7b01d2b21258, ASSIGN in 323 msec 2023-03-05 19:56:02,517 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-05 19:56:02,517 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMinorCompactionWithMonthlyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678046162517"}]},"ts":"1678046162517"} 2023-03-05 19:56:02,518 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testMinorCompactionWithMonthlyPolicy, state=ENABLED in hbase:meta 2023-03-05 19:56:02,521 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_POST_OPERATION 2023-03-05 19:56:02,522 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=112, state=SUCCESS; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy in 1.1550 sec 2023-03-05 19:56:03,474 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-03-05 19:56:03,474 INFO [Listener at localhost/40787] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMinorCompactionWithMonthlyPolicy, procId: 112 completed 2023-03-05 19:56:03,493 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-05 19:56:03,494 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:03,494 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-05 19:56:03,494 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:56:03,494 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-05 19:56:03,495 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-05 19:56:03,495 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:03,495 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:03,496 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:56:03,496 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:03,496 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:56:03,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:56:03,497 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:03,497 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-05 19:56:03,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:03,497 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:03,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-05 19:56:03,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:03,498 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:03,498 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:03,498 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-05 19:56:03,498 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:56:03,498 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-05 19:56:03,498 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-05 19:56:03,498 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-05 19:56:03,498 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1678046161366.014dc240b491a64066ab7b01d2b21258. 2023-03-05 19:56:03,498 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1678046161366.014dc240b491a64066ab7b01d2b21258. started... 2023-03-05 19:56:03,499 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 014dc240b491a64066ab7b01d2b21258 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-03-05 19:56:03,921 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511308d5bfa94d68c4299b78097fa2f750041 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201511308d5bfa94d68c4299b78097fa2f750041 2023-03-05 19:56:04,324 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258/.tmp/family1/1588d6f89ab744748ad1303bac273ab7 2023-03-05 19:56:04,328 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258/.tmp/family1/1588d6f89ab744748ad1303bac273ab7 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258/family1/1588d6f89ab744748ad1303bac273ab7 2023-03-05 19:56:04,332 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258/family1/1588d6f89ab744748ad1303bac273ab7, entries=1000, sequenceid=5, filesize=166.0 K 2023-03-05 19:56:04,333 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 014dc240b491a64066ab7b01d2b21258 in 834ms, sequenceid=5, compaction requested=false 2023-03-05 19:56:04,333 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMinorCompactionWithMonthlyPolicy' 2023-03-05 19:56:04,334 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 014dc240b491a64066ab7b01d2b21258: 2023-03-05 19:56:04,334 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1678046161366.014dc240b491a64066ab7b01d2b21258. 2023-03-05 19:56:04,334 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-05 19:56:04,334 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-05 19:56:04,334 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:04,334 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-05 19:56:04,334 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-05 19:56:04,336 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:04,336 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:04,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:04,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:56:04,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:56:04,336 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:04,336 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-05 19:56:04,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:56:04,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:56:04,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:04,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:04,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:56:04,338 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-05 19:56:04,338 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7685f7df[Count = 0] remaining members to acquire global barrier 2023-03-05 19:56:04,338 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-05 19:56:04,338 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:04,339 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:04,339 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:04,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:04,339 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-05 19:56:04,339 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-05 19:56:04,339 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,37037,1678046018983' in zk 2023-03-05 19:56:04,339 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:04,339 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-05 19:56:04,342 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-05 19:56:04,342 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:56:04,342 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:04,342 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-05 19:56:04,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:04,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:56:04,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:56:04,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:56:04,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:56:04,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:04,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:04,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:56:04,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:04,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:04,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,37037,1678046018983': 2023-03-05 19:56:04,344 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-05 19:56:04,344 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-05 19:56:04,344 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-05 19:56:04,344 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-05 19:56:04,344 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:04,344 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-05 19:56:04,346 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:04,346 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:04,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:04,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:56:04,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:56:04,346 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:04,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:04,346 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:56:04,346 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:56:04,346 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:04,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:56:04,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:56:04,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:04,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:04,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:56:04,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:04,347 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:04,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:04,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:56:04,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:04,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:04,349 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:04,349 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:04,350 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:04,350 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:04,350 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:04,350 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:04,350 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:04,350 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-05 19:56:04,350 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-05 19:56:04,350 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:56:04,350 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:56:04,350 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:56:04,350 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:56:04,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:56:04,350 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-05 19:56:04,350 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-05 19:56:04,350 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-05 19:56:04,350 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:56:04,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:56:04,450 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-05 19:56:04,451 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-05 19:56:04,455 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-05 19:56:04,456 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-05 19:56:04,456 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:04,456 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-05 19:56:04,456 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:56:04,456 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-05 19:56:04,456 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-05 19:56:04,457 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:04,457 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:04,458 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:04,458 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:56:04,458 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:56:04,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:56:04,458 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:04,458 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-05 19:56:04,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:04,459 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:04,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-05 19:56:04,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:04,459 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:04,459 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-03-05 19:56:04,459 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:04,460 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-05 19:56:04,460 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:56:04,460 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-05 19:56:04,460 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-05 19:56:04,460 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-05 19:56:04,460 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1678046161366.014dc240b491a64066ab7b01d2b21258. 2023-03-05 19:56:04,460 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1678046161366.014dc240b491a64066ab7b01d2b21258. started... 2023-03-05 19:56:04,460 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 014dc240b491a64066ab7b01d2b21258 1/1 column families, dataSize=74 B heapSize=408 B 2023-03-05 19:56:04,871 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128a57a746ca2f14c14be3286cb3e8aa12f to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151128a57a746ca2f14c14be3286cb3e8aa12f 2023-03-05 19:56:05,276 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258/.tmp/family1/6d0c40d0fd584d2f834141c7c452bd3c 2023-03-05 19:56:05,280 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258/.tmp/family1/6d0c40d0fd584d2f834141c7c452bd3c as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258/family1/6d0c40d0fd584d2f834141c7c452bd3c 2023-03-05 19:56:05,284 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258/family1/6d0c40d0fd584d2f834141c7c452bd3c, entries=1, sequenceid=9, filesize=5.0 K 2023-03-05 19:56:05,285 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 014dc240b491a64066ab7b01d2b21258 in 825ms, sequenceid=9, compaction requested=false 2023-03-05 19:56:05,285 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 014dc240b491a64066ab7b01d2b21258: 2023-03-05 19:56:05,285 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1678046161366.014dc240b491a64066ab7b01d2b21258. 2023-03-05 19:56:05,285 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-05 19:56:05,285 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-05 19:56:05,285 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:05,285 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-05 19:56:05,285 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-05 19:56:05,288 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:05,288 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:05,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:05,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:56:05,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:56:05,288 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:05,288 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-05 19:56:05,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:56:05,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:56:05,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:05,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:05,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:56:05,289 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-05 19:56:05,289 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@458e0c13[Count = 0] remaining members to acquire global barrier 2023-03-05 19:56:05,289 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-05 19:56:05,289 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:05,291 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:05,291 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:05,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:05,291 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-05 19:56:05,291 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-05 19:56:05,291 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,37037,1678046018983' in zk 2023-03-05 19:56:05,291 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:05,291 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-05 19:56:05,292 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-05 19:56:05,292 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:56:05,292 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:05,292 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-05 19:56:05,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:05,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:56:05,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:56:05,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:56:05,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:56:05,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:05,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:05,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:56:05,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:05,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:05,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,37037,1678046018983': 2023-03-05 19:56:05,294 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-05 19:56:05,294 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-05 19:56:05,294 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-05 19:56:05,294 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-05 19:56:05,294 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:05,294 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-05 19:56:05,296 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:05,296 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:05,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:05,296 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:05,296 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:56:05,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:05,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:56:05,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:56:05,296 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:56:05,296 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:05,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:56:05,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:56:05,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:05,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:05,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:56:05,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:05,298 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:05,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:05,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:56:05,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:05,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:05,300 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:05,300 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:56:05,301 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:56:05,301 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:56:05,301 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-05 19:56:05,301 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:05,301 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:56:05,301 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-05 19:56:05,301 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-05 19:56:05,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:56:05,301 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:05,301 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:05,301 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:05,301 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:05,301 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:05,301 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-05 19:56:05,301 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-05 19:56:05,301 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:56:05,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:56:05,401 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-05 19:56:05,402 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-05 19:56:05,406 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-05 19:56:05,407 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-05 19:56:05,407 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:05,407 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-05 19:56:05,407 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:56:05,407 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-05 19:56:05,407 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-05 19:56:05,408 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:05,408 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:05,413 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:05,413 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:56:05,413 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:56:05,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:56:05,413 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:05,413 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-05 19:56:05,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:05,414 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:05,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-05 19:56:05,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:05,414 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:05,414 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-03-05 19:56:05,414 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:05,414 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-05 19:56:05,414 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:56:05,414 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-05 19:56:05,414 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-05 19:56:05,414 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-05 19:56:05,415 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1678046161366.014dc240b491a64066ab7b01d2b21258. 2023-03-05 19:56:05,415 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1678046161366.014dc240b491a64066ab7b01d2b21258. started... 2023-03-05 19:56:05,415 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 014dc240b491a64066ab7b01d2b21258 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-05 19:56:05,826 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151201f7ecea66109847859eaf85034828ba14 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151201f7ecea66109847859eaf85034828ba14 2023-03-05 19:56:06,232 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258/.tmp/family1/f304fc189dd04dddb574660aa3d652d2 2023-03-05 19:56:06,236 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258/.tmp/family1/f304fc189dd04dddb574660aa3d652d2 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258/family1/f304fc189dd04dddb574660aa3d652d2 2023-03-05 19:56:06,239 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258/family1/f304fc189dd04dddb574660aa3d652d2, entries=1, sequenceid=13, filesize=4.9 K 2023-03-05 19:56:06,239 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 014dc240b491a64066ab7b01d2b21258 in 824ms, sequenceid=13, compaction requested=true 2023-03-05 19:56:06,239 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 014dc240b491a64066ab7b01d2b21258: 2023-03-05 19:56:06,239 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1678046161366.014dc240b491a64066ab7b01d2b21258. 2023-03-05 19:56:06,239 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-05 19:56:06,240 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-05 19:56:06,240 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:06,240 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-05 19:56:06,240 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-05 19:56:06,241 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:06,241 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:06,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:06,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:56:06,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:56:06,241 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:06,241 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-05 19:56:06,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:56:06,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:56:06,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:06,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:06,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:56:06,243 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-05 19:56:06,243 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2bd1048a[Count = 0] remaining members to acquire global barrier 2023-03-05 19:56:06,243 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-05 19:56:06,243 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:06,244 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:06,244 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:06,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:06,244 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-05 19:56:06,244 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:06,244 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-05 19:56:06,244 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-05 19:56:06,244 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,37037,1678046018983' in zk 2023-03-05 19:56:06,246 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:06,246 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-05 19:56:06,247 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:56:06,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:06,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:56:06,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:56:06,247 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-05 19:56:06,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:56:06,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:56:06,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:06,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:06,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:56:06,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:06,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:06,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,37037,1678046018983': 2023-03-05 19:56:06,248 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-05 19:56:06,248 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-05 19:56:06,248 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-05 19:56:06,248 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-05 19:56:06,248 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:06,248 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-05 19:56:06,250 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:06,250 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:06,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:06,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:56:06,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:56:06,250 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:06,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:06,250 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:56:06,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:56:06,250 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:06,251 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:56:06,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:56:06,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:06,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:06,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:56:06,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:06,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:06,251 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:06,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:56:06,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:06,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:06,253 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:06,253 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:56:06,253 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:06,253 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:56:06,254 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-05 19:56:06,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:56:06,253 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-05 19:56:06,254 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-05 19:56:06,253 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:56:06,253 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:06,253 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:56:06,254 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:06,254 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:06,254 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:56:06,254 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-05 19:56:06,254 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-05 19:56:06,254 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:06,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:56:06,254 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:06,354 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-05 19:56:06,355 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-05 19:56:06,358 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-05 19:56:06,360 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-05 19:56:06,360 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:06,360 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-05 19:56:06,360 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:56:06,360 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-05 19:56:06,360 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-05 19:56:06,360 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:06,360 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:06,362 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:06,362 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:56:06,362 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:56:06,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:56:06,362 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:06,362 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-05 19:56:06,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:06,362 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:06,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-05 19:56:06,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:06,362 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:06,363 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-03-05 19:56:06,363 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:06,363 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-05 19:56:06,363 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:56:06,363 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-05 19:56:06,363 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-05 19:56:06,363 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-05 19:56:06,363 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1678046161366.014dc240b491a64066ab7b01d2b21258. 2023-03-05 19:56:06,363 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1678046161366.014dc240b491a64066ab7b01d2b21258. started... 2023-03-05 19:56:06,363 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 014dc240b491a64066ab7b01d2b21258 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-05 19:56:06,774 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205d75d20a6d62d443d9e286bced4755cc0 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151205d75d20a6d62d443d9e286bced4755cc0 2023-03-05 19:56:06,778 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258/.tmp/family1/506dc1adfbe24857aea93a466d91ad03 2023-03-05 19:56:06,783 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258/.tmp/family1/506dc1adfbe24857aea93a466d91ad03 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258/family1/506dc1adfbe24857aea93a466d91ad03 2023-03-05 19:56:06,786 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258/family1/506dc1adfbe24857aea93a466d91ad03, entries=1, sequenceid=17, filesize=4.9 K 2023-03-05 19:56:06,787 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 014dc240b491a64066ab7b01d2b21258 in 424ms, sequenceid=17, compaction requested=true 2023-03-05 19:56:06,787 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 014dc240b491a64066ab7b01d2b21258: 2023-03-05 19:56:06,787 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1678046161366.014dc240b491a64066ab7b01d2b21258. 2023-03-05 19:56:06,788 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-05 19:56:06,788 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-05 19:56:06,788 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:06,788 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-05 19:56:06,788 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-05 19:56:06,790 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:06,790 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:06,790 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-05 19:56:06,791 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:06,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:06,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:56:06,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:56:06,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:56:06,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:56:06,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:06,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:06,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:56:06,793 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-05 19:56:06,793 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3e346d4[Count = 0] remaining members to acquire global barrier 2023-03-05 19:56:06,793 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-05 19:56:06,793 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:06,794 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:06,794 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:06,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:06,794 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-05 19:56:06,794 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-05 19:56:06,794 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,37037,1678046018983' in zk 2023-03-05 19:56:06,794 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:06,794 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-05 19:56:06,797 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:06,797 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-05 19:56:06,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:06,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:56:06,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:56:06,797 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:56:06,797 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-05 19:56:06,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:56:06,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:56:06,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:06,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:06,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:56:06,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:06,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:06,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,37037,1678046018983': 2023-03-05 19:56:06,800 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-05 19:56:06,800 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-05 19:56:06,800 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-05 19:56:06,800 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-05 19:56:06,800 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:06,800 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-05 19:56:06,801 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:06,802 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:06,802 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:06,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:06,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:56:06,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:56:06,802 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:56:06,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:06,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:56:06,802 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:56:06,802 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:06,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:56:06,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:06,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:06,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:56:06,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:06,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:06,803 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:06,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:56:06,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:06,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:06,805 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:56:06,805 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:06,805 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:56:06,805 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:56:06,805 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-05 19:56:06,806 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-05 19:56:06,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:56:06,805 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:06,805 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:56:06,806 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:06,806 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-05 19:56:06,806 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-05 19:56:06,806 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:56:06,806 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:06,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:56:06,806 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:06,806 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-05 19:56:06,806 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:06,806 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:06,906 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-05 19:56:06,907 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-05 19:56:06,910 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-05 19:56:06,911 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-05 19:56:06,911 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:06,911 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-05 19:56:06,912 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:56:06,912 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-05 19:56:06,912 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-05 19:56:06,912 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:06,912 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:06,913 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:56:06,913 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:06,913 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:56:06,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:56:06,913 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:06,913 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-05 19:56:06,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:06,914 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:06,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-05 19:56:06,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:06,914 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:06,914 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-03-05 19:56:06,914 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:06,914 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-05 19:56:06,914 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:56:06,914 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-05 19:56:06,915 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-05 19:56:06,915 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-05 19:56:06,915 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1678046161366.014dc240b491a64066ab7b01d2b21258. 2023-03-05 19:56:06,915 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1678046161366.014dc240b491a64066ab7b01d2b21258. started... 2023-03-05 19:56:06,915 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 014dc240b491a64066ab7b01d2b21258 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-05 19:56:07,325 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151228cd62510c89ca47fc90f7f5d3ff6f6906 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151228cd62510c89ca47fc90f7f5d3ff6f6906 2023-03-05 19:56:07,330 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258/.tmp/family1/af2be8508d08461282a8f57a563cd842 2023-03-05 19:56:07,334 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258/.tmp/family1/af2be8508d08461282a8f57a563cd842 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258/family1/af2be8508d08461282a8f57a563cd842 2023-03-05 19:56:07,338 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258/family1/af2be8508d08461282a8f57a563cd842, entries=1, sequenceid=21, filesize=4.9 K 2023-03-05 19:56:07,339 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 014dc240b491a64066ab7b01d2b21258 in 424ms, sequenceid=21, compaction requested=true 2023-03-05 19:56:07,339 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 014dc240b491a64066ab7b01d2b21258: 2023-03-05 19:56:07,339 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1678046161366.014dc240b491a64066ab7b01d2b21258. 2023-03-05 19:56:07,339 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-05 19:56:07,339 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-05 19:56:07,339 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:07,339 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-05 19:56:07,339 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-05 19:56:07,341 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:07,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:07,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:56:07,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:56:07,341 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:07,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:56:07,342 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:07,342 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-05 19:56:07,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:56:07,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:07,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:07,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:56:07,343 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-05 19:56:07,343 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5a017278[Count = 0] remaining members to acquire global barrier 2023-03-05 19:56:07,343 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-05 19:56:07,343 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:07,344 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:07,344 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:07,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:07,344 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-05 19:56:07,345 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-05 19:56:07,345 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,37037,1678046018983' in zk 2023-03-05 19:56:07,345 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:07,345 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-05 19:56:07,346 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-05 19:56:07,346 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:56:07,346 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:07,346 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-05 19:56:07,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:07,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:56:07,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:56:07,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:56:07,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:56:07,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:07,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:07,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:56:07,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:07,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:07,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,37037,1678046018983': 2023-03-05 19:56:07,348 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-05 19:56:07,348 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-05 19:56:07,348 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-05 19:56:07,348 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-05 19:56:07,348 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:07,348 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-05 19:56:07,353 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:07,353 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:07,353 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:07,353 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:56:07,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:07,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:07,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:56:07,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:56:07,353 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:56:07,353 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:07,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:56:07,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:56:07,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:07,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:07,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:56:07,354 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:07,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:07,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:07,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:56:07,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:07,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:07,359 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:07,359 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:56:07,359 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:56:07,359 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:07,359 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:56:07,359 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:56:07,359 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-05 19:56:07,359 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-05 19:56:07,359 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-05 19:56:07,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:56:07,359 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:07,359 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:07,359 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:07,359 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:07,359 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-05 19:56:07,359 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:07,359 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:56:07,359 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-05 19:56:07,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:56:07,460 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-05 19:56:07,460 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-05 19:56:07,464 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-05 19:56:07,465 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-05 19:56:07,465 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:07,465 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-05 19:56:07,465 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:56:07,465 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-05 19:56:07,465 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-05 19:56:07,466 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:07,466 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:07,467 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:56:07,467 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:07,467 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:56:07,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:56:07,467 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:07,467 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-05 19:56:07,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:07,468 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:07,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-05 19:56:07,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:07,468 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:07,468 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-03-05 19:56:07,468 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:07,468 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-05 19:56:07,468 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:56:07,468 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-05 19:56:07,468 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-05 19:56:07,468 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-05 19:56:07,468 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1678046161366.014dc240b491a64066ab7b01d2b21258. 2023-03-05 19:56:07,469 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1678046161366.014dc240b491a64066ab7b01d2b21258. started... 2023-03-05 19:56:07,469 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 014dc240b491a64066ab7b01d2b21258 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-05 19:56:07,533 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-05 19:56:07,534 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x096139a4 to 127.0.0.1:62957 2023-03-05 19:56:07,534 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-05 19:56:07,534 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-05 19:56:07,534 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x3894c415 to 127.0.0.1:62957 2023-03-05 19:56:07,534 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-05 19:56:07,535 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-05 19:56:07,535 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x375aed01 to 127.0.0.1:62957 2023-03-05 19:56:07,535 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-05 19:56:07,879 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015123186464ae98a9e44ccab33cc46ce38eae0 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015123186464ae98a9e44ccab33cc46ce38eae0 2023-03-05 19:56:07,884 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258/.tmp/family1/1cf8127deea846d896b49416290cbe06 2023-03-05 19:56:07,887 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258/.tmp/family1/1cf8127deea846d896b49416290cbe06 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258/family1/1cf8127deea846d896b49416290cbe06 2023-03-05 19:56:07,891 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258/family1/1cf8127deea846d896b49416290cbe06, entries=1, sequenceid=25, filesize=4.9 K 2023-03-05 19:56:07,892 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 014dc240b491a64066ab7b01d2b21258 in 423ms, sequenceid=25, compaction requested=true 2023-03-05 19:56:07,892 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 014dc240b491a64066ab7b01d2b21258: 2023-03-05 19:56:07,892 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1678046161366.014dc240b491a64066ab7b01d2b21258. 2023-03-05 19:56:07,892 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-05 19:56:07,892 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-05 19:56:07,892 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:07,892 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-05 19:56:07,892 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-05 19:56:07,895 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:07,895 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:07,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:07,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:56:07,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:56:07,895 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:07,895 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-05 19:56:07,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:56:07,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:56:07,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:07,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:07,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:56:07,897 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-05 19:56:07,898 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6f5a13bf[Count = 0] remaining members to acquire global barrier 2023-03-05 19:56:07,898 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-05 19:56:07,898 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:07,901 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:07,901 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:07,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:07,901 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-05 19:56:07,901 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-05 19:56:07,901 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:07,901 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-05 19:56:07,901 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,37037,1678046018983' in zk 2023-03-05 19:56:07,905 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-05 19:56:07,905 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:56:07,905 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-05 19:56:07,906 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:07,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:07,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:56:07,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:56:07,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:56:07,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:56:07,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:07,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:07,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:56:07,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:07,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:07,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,37037,1678046018983': 2023-03-05 19:56:07,909 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-05 19:56:07,909 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-05 19:56:07,909 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-05 19:56:07,909 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-05 19:56:07,909 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:07,909 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-05 19:56:07,910 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:07,910 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:07,910 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:07,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:07,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:56:07,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:56:07,910 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:56:07,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:07,911 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:07,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:56:07,911 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:56:07,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:56:07,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:07,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:07,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:56:07,912 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:07,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:07,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:07,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:56:07,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:07,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:07,914 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:07,914 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:07,914 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:07,914 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:56:07,914 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-05 19:56:07,914 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-05 19:56:07,914 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:07,914 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:07,914 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:07,914 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:07,914 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:56:07,915 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:56:07,915 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-05 19:56:07,915 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:56:07,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:56:07,915 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-05 19:56:07,915 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-05 19:56:07,915 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:56:07,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:56:08,015 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-05 19:56:08,016 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-05 19:56:08,020 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-05 19:56:08,026 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-05 19:56:08,026 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:08,026 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-05 19:56:08,026 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:56:08,026 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-05 19:56:08,026 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-05 19:56:08,026 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:08,026 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:08,027 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:56:08,027 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:08,028 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:56:08,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:56:08,028 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:08,028 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-05 19:56:08,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:08,028 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:08,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-05 19:56:08,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:08,029 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:08,029 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-03-05 19:56:08,029 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:08,029 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-05 19:56:08,029 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:56:08,029 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-05 19:56:08,029 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-05 19:56:08,029 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-05 19:56:08,029 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1678046161366.014dc240b491a64066ab7b01d2b21258. 2023-03-05 19:56:08,029 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1678046161366.014dc240b491a64066ab7b01d2b21258. started... 2023-03-05 19:56:08,029 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 014dc240b491a64066ab7b01d2b21258 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-05 19:56:08,063 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-05 19:56:08,441 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601011b009546899f408b84675087ea8affec to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201601011b009546899f408b84675087ea8affec 2023-03-05 19:56:08,845 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258/.tmp/family1/23363596f8fc4101a11689d2a9c94d59 2023-03-05 19:56:08,849 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258/.tmp/family1/23363596f8fc4101a11689d2a9c94d59 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258/family1/23363596f8fc4101a11689d2a9c94d59 2023-03-05 19:56:08,852 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258/family1/23363596f8fc4101a11689d2a9c94d59, entries=1, sequenceid=29, filesize=4.9 K 2023-03-05 19:56:08,852 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 014dc240b491a64066ab7b01d2b21258 in 823ms, sequenceid=29, compaction requested=true 2023-03-05 19:56:08,852 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 014dc240b491a64066ab7b01d2b21258: 2023-03-05 19:56:08,852 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1678046161366.014dc240b491a64066ab7b01d2b21258. 2023-03-05 19:56:08,852 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-05 19:56:08,853 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-05 19:56:08,853 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:08,853 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-05 19:56:08,853 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-05 19:56:08,855 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:08,855 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:08,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:08,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:56:08,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:56:08,855 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:08,855 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-05 19:56:08,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:56:08,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:56:08,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:08,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:08,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:56:08,856 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-05 19:56:08,856 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4c8cbf8a[Count = 0] remaining members to acquire global barrier 2023-03-05 19:56:08,856 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-05 19:56:08,857 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:08,858 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:08,858 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:08,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:08,858 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-05 19:56:08,858 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-05 19:56:08,858 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,37037,1678046018983' in zk 2023-03-05 19:56:08,858 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:08,858 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-05 19:56:08,859 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-05 19:56:08,859 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:08,859 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:56:08,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:08,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:56:08,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:56:08,859 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-05 19:56:08,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:56:08,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:56:08,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:08,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:08,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:56:08,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:08,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:08,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,37037,1678046018983': 2023-03-05 19:56:08,861 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-05 19:56:08,861 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-05 19:56:08,861 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-05 19:56:08,861 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-05 19:56:08,861 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:08,861 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-05 19:56:08,862 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:08,862 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:08,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:08,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:56:08,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:56:08,862 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:08,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:08,862 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:56:08,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:56:08,863 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:08,863 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:56:08,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:56:08,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:08,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:08,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:56:08,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:08,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:08,864 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:08,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:56:08,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:08,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:08,866 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:08,866 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:56:08,866 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:08,866 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:56:08,866 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-05 19:56:08,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:56:08,866 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-05 19:56:08,866 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-05 19:56:08,866 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:08,866 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:56:08,866 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:56:08,866 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:08,867 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:08,867 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:08,867 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:08,867 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-05 19:56:08,867 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:56:08,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:56:08,867 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-05 19:56:08,967 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-05 19:56:08,968 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-05 19:56:08,971 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-05 19:56:08,973 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-05 19:56:08,973 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:08,973 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-05 19:56:08,973 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:56:08,973 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-05 19:56:08,973 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-05 19:56:08,973 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:08,973 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:08,974 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:08,974 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:56:08,974 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:56:08,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:56:08,975 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:08,975 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-05 19:56:08,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:08,975 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:08,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-05 19:56:08,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:08,975 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:08,975 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-03-05 19:56:08,976 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:08,976 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-05 19:56:08,976 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:56:08,976 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-05 19:56:08,976 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-05 19:56:08,976 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-05 19:56:08,976 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1678046161366.014dc240b491a64066ab7b01d2b21258. 2023-03-05 19:56:08,976 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1678046161366.014dc240b491a64066ab7b01d2b21258. started... 2023-03-05 19:56:08,976 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 014dc240b491a64066ab7b01d2b21258 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-05 19:56:09,389 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601033c24ba7e607247b492a14a41404e2a73 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201601033c24ba7e607247b492a14a41404e2a73 2023-03-05 19:56:09,793 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258/.tmp/family1/be6a140ca6f747f8ae2711150c072e9d 2023-03-05 19:56:09,797 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258/.tmp/family1/be6a140ca6f747f8ae2711150c072e9d as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258/family1/be6a140ca6f747f8ae2711150c072e9d 2023-03-05 19:56:09,800 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258/family1/be6a140ca6f747f8ae2711150c072e9d, entries=1, sequenceid=33, filesize=4.9 K 2023-03-05 19:56:09,800 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 014dc240b491a64066ab7b01d2b21258 in 824ms, sequenceid=33, compaction requested=true 2023-03-05 19:56:09,800 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 014dc240b491a64066ab7b01d2b21258: 2023-03-05 19:56:09,800 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1678046161366.014dc240b491a64066ab7b01d2b21258. 2023-03-05 19:56:09,801 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-05 19:56:09,801 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-05 19:56:09,801 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:09,801 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-05 19:56:09,801 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-05 19:56:09,802 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:09,802 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:09,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:09,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:56:09,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:56:09,803 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:09,803 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-05 19:56:09,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:56:09,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:56:09,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:09,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:09,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:56:09,804 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-05 19:56:09,804 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@55408529[Count = 0] remaining members to acquire global barrier 2023-03-05 19:56:09,804 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-05 19:56:09,804 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:09,805 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:09,805 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:09,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:09,805 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-05 19:56:09,805 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-05 19:56:09,805 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,37037,1678046018983' in zk 2023-03-05 19:56:09,805 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:09,805 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-05 19:56:09,807 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-05 19:56:09,807 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:09,807 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:56:09,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:09,807 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-05 19:56:09,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:56:09,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:56:09,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:56:09,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:56:09,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:09,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:09,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:56:09,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:09,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:09,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,37037,1678046018983': 2023-03-05 19:56:09,809 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-05 19:56:09,809 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-05 19:56:09,809 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-05 19:56:09,809 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-05 19:56:09,809 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:09,809 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-05 19:56:09,818 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:09,818 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:09,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:09,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:56:09,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:56:09,818 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:09,818 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:56:09,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:09,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:56:09,818 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:56:09,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:56:09,818 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:09,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:09,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:09,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:56:09,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:09,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:09,819 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:09,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:56:09,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:09,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:09,821 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:09,821 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:56:09,821 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:09,821 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:56:09,821 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-05 19:56:09,821 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-05 19:56:09,821 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:09,821 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:56:09,821 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:56:09,821 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:09,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:56:09,822 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-05 19:56:09,822 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:09,822 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-05 19:56:09,822 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:09,822 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:56:09,822 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:09,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:56:09,922 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-05 19:56:09,923 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-05 19:56:09,926 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-05 19:56:09,927 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-05 19:56:09,927 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:09,927 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-05 19:56:09,927 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:56:09,927 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-05 19:56:09,927 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-05 19:56:09,928 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:09,928 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:09,929 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:09,929 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:56:09,929 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:56:09,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:56:09,929 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:09,929 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-05 19:56:09,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:09,930 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:09,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-05 19:56:09,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:09,930 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:09,930 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-03-05 19:56:09,930 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:09,930 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-05 19:56:09,930 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:56:09,930 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-05 19:56:09,930 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-05 19:56:09,930 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-05 19:56:09,930 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1678046161366.014dc240b491a64066ab7b01d2b21258. 2023-03-05 19:56:09,931 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1678046161366.014dc240b491a64066ab7b01d2b21258. started... 2023-03-05 19:56:09,931 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 014dc240b491a64066ab7b01d2b21258 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-05 19:56:10,341 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20150907c691201b38e242d28dfa46cca0b8d03c to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20150907c691201b38e242d28dfa46cca0b8d03c 2023-03-05 19:56:10,745 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258/.tmp/family1/18a8688e9d594ee489e76f0acc674ed1 2023-03-05 19:56:10,749 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258/.tmp/family1/18a8688e9d594ee489e76f0acc674ed1 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258/family1/18a8688e9d594ee489e76f0acc674ed1 2023-03-05 19:56:10,752 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258/family1/18a8688e9d594ee489e76f0acc674ed1, entries=1, sequenceid=37, filesize=4.9 K 2023-03-05 19:56:10,752 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 014dc240b491a64066ab7b01d2b21258 in 821ms, sequenceid=37, compaction requested=true 2023-03-05 19:56:10,753 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 014dc240b491a64066ab7b01d2b21258: 2023-03-05 19:56:10,753 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1678046161366.014dc240b491a64066ab7b01d2b21258. 2023-03-05 19:56:10,753 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-05 19:56:10,753 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-05 19:56:10,753 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:10,753 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-05 19:56:10,753 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-05 19:56:10,756 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:10,756 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:10,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:10,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:56:10,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:56:10,756 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:10,756 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-05 19:56:10,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:56:10,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:56:10,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:10,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:10,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:56:10,757 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-05 19:56:10,757 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@403af4b[Count = 0] remaining members to acquire global barrier 2023-03-05 19:56:10,757 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-05 19:56:10,757 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:10,758 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:10,758 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:10,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:10,759 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-05 19:56:10,759 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-05 19:56:10,759 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,37037,1678046018983' in zk 2023-03-05 19:56:10,759 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:10,759 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-05 19:56:10,760 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-05 19:56:10,760 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:10,760 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:56:10,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:10,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:56:10,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:56:10,760 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-05 19:56:10,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:56:10,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:56:10,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:10,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:10,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:56:10,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:10,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:10,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,37037,1678046018983': 2023-03-05 19:56:10,762 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-05 19:56:10,762 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-05 19:56:10,762 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-05 19:56:10,762 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-05 19:56:10,762 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:10,762 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-05 19:56:10,763 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:10,763 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:10,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:10,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:56:10,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:56:10,763 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:10,763 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:56:10,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:10,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:56:10,764 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:10,764 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:56:10,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:56:10,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:10,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:10,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:56:10,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:10,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:10,765 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:10,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:56:10,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:10,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:10,767 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:10,767 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:56:10,767 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:10,768 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:56:10,768 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-05 19:56:10,768 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-05 19:56:10,768 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:56:10,768 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:10,768 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:56:10,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:56:10,768 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:10,768 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-05 19:56:10,768 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:10,768 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:10,768 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:10,768 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:56:10,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:56:10,768 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-05 19:56:10,768 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-05 19:56:10,868 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-05 19:56:10,869 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-05 19:56:10,873 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-05 19:56:10,874 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-05 19:56:10,874 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:10,874 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-05 19:56:10,874 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:56:10,874 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-05 19:56:10,874 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-05 19:56:10,874 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:10,874 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:10,876 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:10,876 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:56:10,876 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:56:10,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:56:10,876 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:10,876 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-05 19:56:10,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:10,876 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:10,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-05 19:56:10,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:10,877 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:10,877 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-03-05 19:56:10,877 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:10,877 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-05 19:56:10,877 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:56:10,877 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-05 19:56:10,877 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-05 19:56:10,877 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-05 19:56:10,877 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1678046161366.014dc240b491a64066ab7b01d2b21258. 2023-03-05 19:56:10,877 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1678046161366.014dc240b491a64066ab7b01d2b21258. started... 2023-03-05 19:56:10,877 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 014dc240b491a64066ab7b01d2b21258 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-05 19:56:11,288 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151120711d81b6e315496c9a26b35a4e882239 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151120711d81b6e315496c9a26b35a4e882239 2023-03-05 19:56:11,693 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258/.tmp/family1/dbe11b8b4ddf4210a8e378a8d3fc8934 2023-03-05 19:56:11,697 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258/.tmp/family1/dbe11b8b4ddf4210a8e378a8d3fc8934 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258/family1/dbe11b8b4ddf4210a8e378a8d3fc8934 2023-03-05 19:56:11,700 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258/family1/dbe11b8b4ddf4210a8e378a8d3fc8934, entries=1, sequenceid=41, filesize=4.9 K 2023-03-05 19:56:11,700 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 014dc240b491a64066ab7b01d2b21258 in 823ms, sequenceid=41, compaction requested=true 2023-03-05 19:56:11,700 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 014dc240b491a64066ab7b01d2b21258: 2023-03-05 19:56:11,700 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1678046161366.014dc240b491a64066ab7b01d2b21258. 2023-03-05 19:56:11,701 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-05 19:56:11,701 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-05 19:56:11,701 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:11,701 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-05 19:56:11,701 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-05 19:56:11,702 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:11,702 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:11,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:11,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:56:11,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:56:11,702 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:11,702 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-05 19:56:11,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:56:11,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:56:11,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:11,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:11,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:56:11,703 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-05 19:56:11,703 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3b7b939f[Count = 0] remaining members to acquire global barrier 2023-03-05 19:56:11,703 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-05 19:56:11,703 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:11,704 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:11,705 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:11,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:11,705 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-05 19:56:11,705 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-05 19:56:11,705 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:11,705 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-05 19:56:11,705 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,37037,1678046018983' in zk 2023-03-05 19:56:11,706 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-05 19:56:11,706 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:11,706 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:56:11,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:11,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:56:11,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:56:11,706 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-05 19:56:11,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:56:11,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:56:11,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:11,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:11,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:56:11,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:11,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:11,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,37037,1678046018983': 2023-03-05 19:56:11,708 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-05 19:56:11,708 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-05 19:56:11,708 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-05 19:56:11,708 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-05 19:56:11,708 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:11,708 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-05 19:56:11,710 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:11,710 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:11,710 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:11,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:11,710 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:56:11,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:11,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:56:11,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:56:11,711 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:56:11,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:56:11,711 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:11,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:56:11,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:11,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:11,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:56:11,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:11,712 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:11,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:11,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:56:11,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:11,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:11,714 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:11,714 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:56:11,714 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:11,714 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:56:11,714 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-05 19:56:11,714 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-05 19:56:11,714 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:11,714 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:56:11,714 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:56:11,715 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-05 19:56:11,715 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:11,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:56:11,715 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-05 19:56:11,715 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:11,715 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-05 19:56:11,715 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:56:11,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:56:11,715 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:11,715 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:11,815 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-05 19:56:11,816 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-05 19:56:11,817 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=38221] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMinorCompactionWithMonthlyPolicy 2023-03-05 19:56:11,818 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] procedure2.ProcedureExecutor(1029): Stored pid=115, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-03-05 19:56:11,819 DEBUG [PEWorker-5] locking.LockProcedure(309): LOCKED pid=115, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-03-05 19:56:11,820 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] mob.MobUtils(833): start MOB compaction of files for table='testMinorCompactionWithMonthlyPolicy', column='family1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-05 19:56:11,820 INFO [PEWorker-5] procedure2.TimeoutExecutorThread(81): ADDED pid=115, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678046771820 2023-03-05 19:56:11,821 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-03-05 19:56:11,822 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMinorCompactionWithMonthlyPolicy' and column 'family1' 2023-03-05 19:56:11,822 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-03-05 19:56:11,822 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] zookeeper.ReadOnlyZKClient(139): Connect 0x7048fa16 to 127.0.0.1:62957 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-05 19:56:11,826 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4157d404, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-05 19:56:11,826 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-03-05 19:56:12,251 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601033552f0cf13b64c20b790a79a9d2982c7 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201601033552f0cf13b64c20b790a79a9d2982c7 2023-03-05 19:56:12,251 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20160101 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-03-05 19:56:12,269 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-05 19:56:12,272 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:40894, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-05 19:56:12,277 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-05 19:56:12,278 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:59640, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-05 19:56:12,285 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/2657d4bca9504469b2b3e1a71855c3df first=Optional[r5] last=Optional[r6] 2023-03-05 19:56:12,286 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMinorCompactionWithMonthlyPolicy,,1678046161366.014dc240b491a64066ab7b01d2b21258., hostname=jenkins-hbase4.apache.org,37037,1678046018983, seqNum=2 for row with hfile group [{family1,hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/2657d4bca9504469b2b3e1a71855c3df}] 2023-03-05 19:56:12,290 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.HStore(763): Validating hfile at hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/2657d4bca9504469b2b3e1a71855c3df for inclusion in 014dc240b491a64066ab7b01d2b21258/family1 2023-03-05 19:56:12,292 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.HStore(775): HFile bounds: first=r5 last=r6 2023-03-05 19:56:12,292 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.HStore(777): Region bounds: first= last= 2023-03-05 19:56:12,292 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.HRegion(2446): Flush status journal for 014dc240b491a64066ab7b01d2b21258: 2023-03-05 19:56:12,293 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/2657d4bca9504469b2b3e1a71855c3df to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testMinorCompactionWithMonthlyPolicy__rcmki54na64mp50nq8unbpb191jqeidm5vit9fcq3nqhb7nnu40acgc4804f5ak9/family1/2657d4bca9504469b2b3e1a71855c3df 2023-03-05 19:56:12,295 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testMinorCompactionWithMonthlyPolicy__rcmki54na64mp50nq8unbpb191jqeidm5vit9fcq3nqhb7nnu40acgc4804f5ak9/family1/2657d4bca9504469b2b3e1a71855c3df as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258/family1/c0e615a9d6a14a7d8b6b84d8b0305c28_SeqId_44_ 2023-03-05 19:56:12,295 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.HStore(849): Loaded HFile hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testMinorCompactionWithMonthlyPolicy__rcmki54na64mp50nq8unbpb191jqeidm5vit9fcq3nqhb7nnu40acgc4804f5ak9/family1/2657d4bca9504469b2b3e1a71855c3df into 014dc240b491a64066ab7b01d2b21258/family1 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258/family1/c0e615a9d6a14a7d8b6b84d8b0305c28_SeqId_44_ - updating store file list. 2023-03-05 19:56:12,299 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.HStore(882): Loaded HFile hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258/family1/c0e615a9d6a14a7d8b6b84d8b0305c28_SeqId_44_ into 014dc240b491a64066ab7b01d2b21258/family1 2023-03-05 19:56:12,299 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.HStore(855): Successfully loaded hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testMinorCompactionWithMonthlyPolicy__rcmki54na64mp50nq8unbpb191jqeidm5vit9fcq3nqhb7nnu40acgc4804f5ak9/family1/2657d4bca9504469b2b3e1a71855c3df into 014dc240b491a64066ab7b01d2b21258/family1 (new location: hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258/family1/c0e615a9d6a14a7d8b6b84d8b0305c28_SeqId_44_) 2023-03-05 19:56:12,300 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testMinorCompactionWithMonthlyPolicy__rcmki54na64mp50nq8unbpb191jqeidm5vit9fcq3nqhb7nnu40acgc4804f5ak9/family1/2657d4bca9504469b2b3e1a71855c3df 2023-03-05 19:56:12,300 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-03-05 19:56:12,302 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 50903 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-03-05 19:56:12,303 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.HStore(1912): 014dc240b491a64066ab7b01d2b21258/family1 is initiating minor compaction 2023-03-05 19:56:12,303 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@6dd72974; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-05 19:56:12,303 INFO [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 014dc240b491a64066ab7b01d2b21258/family1 in testMinorCompactionWithMonthlyPolicy,,1678046161366.014dc240b491a64066ab7b01d2b21258. 2023-03-05 19:56:12,303 INFO [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258/family1/6d0c40d0fd584d2f834141c7c452bd3c, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258/family1/f304fc189dd04dddb574660aa3d652d2, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258/family1/506dc1adfbe24857aea93a466d91ad03, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258/family1/af2be8508d08461282a8f57a563cd842, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258/family1/1cf8127deea846d896b49416290cbe06, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258/family1/23363596f8fc4101a11689d2a9c94d59, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258/family1/be6a140ca6f747f8ae2711150c072e9d, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258/family1/18a8688e9d594ee489e76f0acc674ed1, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258/family1/dbe11b8b4ddf4210a8e378a8d3fc8934, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258/family1/c0e615a9d6a14a7d8b6b84d8b0305c28_SeqId_44_] into tmpdir=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258/.tmp, totalSize=49.7 K 2023-03-05 19:56:12,303 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.Compactor(207): Compacting 6d0c40d0fd584d2f834141c7c452bd3c, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-03-05 19:56:12,303 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.Compactor(207): Compacting f304fc189dd04dddb574660aa3d652d2, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=13 2023-03-05 19:56:12,303 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.Compactor(207): Compacting 506dc1adfbe24857aea93a466d91ad03, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=17 2023-03-05 19:56:12,303 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.Compactor(207): Compacting af2be8508d08461282a8f57a563cd842, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=21 2023-03-05 19:56:12,303 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.Compactor(207): Compacting 1cf8127deea846d896b49416290cbe06, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=25 2023-03-05 19:56:12,303 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.Compactor(207): Compacting 23363596f8fc4101a11689d2a9c94d59, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=29 2023-03-05 19:56:12,303 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.Compactor(207): Compacting be6a140ca6f747f8ae2711150c072e9d, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=33 2023-03-05 19:56:12,303 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.Compactor(207): Compacting 18a8688e9d594ee489e76f0acc674ed1, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=37 2023-03-05 19:56:12,303 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.Compactor(207): Compacting dbe11b8b4ddf4210a8e378a8d3fc8934, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=41 2023-03-05 19:56:12,303 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.Compactor(207): Compacting c0e615a9d6a14a7d8b6b84d8b0305c28_SeqId_44_, keycount=2, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=45 2023-03-05 19:56:12,305 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20160101 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-03-05 19:56:12,307 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-05 19:56:12,309 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201601011b009546899f408b84675087ea8affec to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201601011b009546899f408b84675087ea8affec 2023-03-05 19:56:12,310 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201601033c24ba7e607247b492a14a41404e2a73 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201601033c24ba7e607247b492a14a41404e2a73 2023-03-05 19:56:12,310 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithMonthlyPolicy 2 expired mob files are deleted 2023-03-05 19:56:12,310 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101' 2023-03-05 19:56:12,311 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-03-05 19:56:12,311 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-03-05 19:56:12,337 INFO [RS:0;jenkins-hbase4:37037-shortCompactions-0] throttle.PressureAwareThroughputController(145): 014dc240b491a64066ab7b01d2b21258#family1#compaction#165 average throughput is 1.37 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-05 19:56:12,760 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015123106a7b9fdf12d46e0970351a6e452812f to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015123106a7b9fdf12d46e0970351a6e452812f 2023-03-05 19:56:12,761 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151201 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-03-05 19:56:12,785 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/d07978405b9e45c8935551c1c9045074 first=Optional[r1] last=Optional[r4] 2023-03-05 19:56:12,786 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMinorCompactionWithMonthlyPolicy,,1678046161366.014dc240b491a64066ab7b01d2b21258., hostname=jenkins-hbase4.apache.org,37037,1678046018983, seqNum=2 for row with hfile group [{family1,hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/d07978405b9e45c8935551c1c9045074}] 2023-03-05 19:56:12,788 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.HStore(763): Validating hfile at hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/d07978405b9e45c8935551c1c9045074 for inclusion in 014dc240b491a64066ab7b01d2b21258/family1 2023-03-05 19:56:12,790 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.HStore(775): HFile bounds: first=r1 last=r4 2023-03-05 19:56:12,790 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.HStore(777): Region bounds: first= last= 2023-03-05 19:56:12,790 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.HRegion(2446): Flush status journal for 014dc240b491a64066ab7b01d2b21258: 2023-03-05 19:56:12,791 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/d07978405b9e45c8935551c1c9045074 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testMinorCompactionWithMonthlyPolicy__hm2ce4pm3dss0djv8pnnk3eisad1isflmjuhi54sd939093lsn7dk19pfmpaum67/family1/d07978405b9e45c8935551c1c9045074 2023-03-05 19:56:12,793 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testMinorCompactionWithMonthlyPolicy__hm2ce4pm3dss0djv8pnnk3eisad1isflmjuhi54sd939093lsn7dk19pfmpaum67/family1/d07978405b9e45c8935551c1c9045074 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258/family1/e50a05d3a69e41b09fc18d18b943c8ea_SeqId_46_ 2023-03-05 19:56:12,793 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.HStore(849): Loaded HFile hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testMinorCompactionWithMonthlyPolicy__hm2ce4pm3dss0djv8pnnk3eisad1isflmjuhi54sd939093lsn7dk19pfmpaum67/family1/d07978405b9e45c8935551c1c9045074 into 014dc240b491a64066ab7b01d2b21258/family1 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258/family1/e50a05d3a69e41b09fc18d18b943c8ea_SeqId_46_ - updating store file list. 2023-03-05 19:56:12,796 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.HStore(882): Loaded HFile hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258/family1/e50a05d3a69e41b09fc18d18b943c8ea_SeqId_46_ into 014dc240b491a64066ab7b01d2b21258/family1 2023-03-05 19:56:12,796 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.HStore(855): Successfully loaded hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testMinorCompactionWithMonthlyPolicy__hm2ce4pm3dss0djv8pnnk3eisad1isflmjuhi54sd939093lsn7dk19pfmpaum67/family1/d07978405b9e45c8935551c1c9045074 into 014dc240b491a64066ab7b01d2b21258/family1 (new location: hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258/family1/e50a05d3a69e41b09fc18d18b943c8ea_SeqId_46_) 2023-03-05 19:56:12,796 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testMinorCompactionWithMonthlyPolicy__hm2ce4pm3dss0djv8pnnk3eisad1isflmjuhi54sd939093lsn7dk19pfmpaum67/family1/d07978405b9e45c8935551c1c9045074 2023-03-05 19:56:12,798 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151201 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-03-05 19:56:12,798 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-05 19:56:12,800 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151201f7ecea66109847859eaf85034828ba14 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151201f7ecea66109847859eaf85034828ba14 2023-03-05 19:56:12,801 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151205d75d20a6d62d443d9e286bced4755cc0 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151205d75d20a6d62d443d9e286bced4755cc0 2023-03-05 19:56:12,801 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151228cd62510c89ca47fc90f7f5d3ff6f6906 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151228cd62510c89ca47fc90f7f5d3ff6f6906 2023-03-05 19:56:12,802 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015123186464ae98a9e44ccab33cc46ce38eae0 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015123186464ae98a9e44ccab33cc46ce38eae0 2023-03-05 19:56:12,802 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithMonthlyPolicy 4 expired mob files are deleted 2023-03-05 19:56:12,802 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201' 2023-03-05 19:56:12,803 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-03-05 19:56:12,803 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-03-05 19:56:13,154 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258/.tmp/family1/be7b46be5d474303a5262cc09dd652dd as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258/family1/be7b46be5d474303a5262cc09dd652dd 2023-03-05 19:56:13,158 INFO [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 014dc240b491a64066ab7b01d2b21258/family1 of 014dc240b491a64066ab7b01d2b21258 into be7b46be5d474303a5262cc09dd652dd(size=6.6 K), total size for store is 178.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-05 19:56:13,158 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 014dc240b491a64066ab7b01d2b21258: 2023-03-05 19:56:13,158 INFO [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMinorCompactionWithMonthlyPolicy,,1678046161366.014dc240b491a64066ab7b01d2b21258., storeName=family1, fileCount=10, fileSize=49.7 K (5.0 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 5.2 K), priority=2, time=1678046172303; duration=0sec 2023-03-05 19:56:13,158 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-05 19:56:13,181 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258/family1/6d0c40d0fd584d2f834141c7c452bd3c, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258/family1/f304fc189dd04dddb574660aa3d652d2, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258/family1/506dc1adfbe24857aea93a466d91ad03, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258/family1/af2be8508d08461282a8f57a563cd842, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258/family1/1cf8127deea846d896b49416290cbe06, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258/family1/23363596f8fc4101a11689d2a9c94d59, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258/family1/be6a140ca6f747f8ae2711150c072e9d, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258/family1/18a8688e9d594ee489e76f0acc674ed1, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258/family1/dbe11b8b4ddf4210a8e378a8d3fc8934, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258/family1/c0e615a9d6a14a7d8b6b84d8b0305c28_SeqId_44_] to archive 2023-03-05 19:56:13,182 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-05 19:56:13,184 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258/family1/6d0c40d0fd584d2f834141c7c452bd3c to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258/family1/6d0c40d0fd584d2f834141c7c452bd3c 2023-03-05 19:56:13,185 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258/family1/f304fc189dd04dddb574660aa3d652d2 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258/family1/f304fc189dd04dddb574660aa3d652d2 2023-03-05 19:56:13,186 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258/family1/506dc1adfbe24857aea93a466d91ad03 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258/family1/506dc1adfbe24857aea93a466d91ad03 2023-03-05 19:56:13,187 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258/family1/af2be8508d08461282a8f57a563cd842 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258/family1/af2be8508d08461282a8f57a563cd842 2023-03-05 19:56:13,188 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258/family1/1cf8127deea846d896b49416290cbe06 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258/family1/1cf8127deea846d896b49416290cbe06 2023-03-05 19:56:13,189 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258/family1/23363596f8fc4101a11689d2a9c94d59 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258/family1/23363596f8fc4101a11689d2a9c94d59 2023-03-05 19:56:13,190 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258/family1/be6a140ca6f747f8ae2711150c072e9d to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258/family1/be6a140ca6f747f8ae2711150c072e9d 2023-03-05 19:56:13,191 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258/family1/18a8688e9d594ee489e76f0acc674ed1 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258/family1/18a8688e9d594ee489e76f0acc674ed1 2023-03-05 19:56:13,192 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258/family1/dbe11b8b4ddf4210a8e378a8d3fc8934 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258/family1/dbe11b8b4ddf4210a8e378a8d3fc8934 2023-03-05 19:56:13,193 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258/family1/c0e615a9d6a14a7d8b6b84d8b0305c28_SeqId_44_ to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258/family1/c0e615a9d6a14a7d8b6b84d8b0305c28_SeqId_44_ 2023-03-05 19:56:13,637 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130366ce25eb4184e5897faf12dcbcc2535 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151130366ce25eb4184e5897faf12dcbcc2535 2023-03-05 19:56:13,638 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-03-05 19:56:13,672 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/ddfd0c29d15247d08eb9cb207da3d9eb first=Optional[r00] last=Optional[r8] 2023-03-05 19:56:13,672 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMinorCompactionWithMonthlyPolicy,,1678046161366.014dc240b491a64066ab7b01d2b21258., hostname=jenkins-hbase4.apache.org,37037,1678046018983, seqNum=2 for row with hfile group [{family1,hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/ddfd0c29d15247d08eb9cb207da3d9eb}] 2023-03-05 19:56:13,675 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.HStore(763): Validating hfile at hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/ddfd0c29d15247d08eb9cb207da3d9eb for inclusion in 014dc240b491a64066ab7b01d2b21258/family1 2023-03-05 19:56:13,679 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.HStore(775): HFile bounds: first=r00 last=r8 2023-03-05 19:56:13,679 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.HStore(777): Region bounds: first= last= 2023-03-05 19:56:13,679 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.HRegion(2446): Flush status journal for 014dc240b491a64066ab7b01d2b21258: 2023-03-05 19:56:13,680 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/ddfd0c29d15247d08eb9cb207da3d9eb to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testMinorCompactionWithMonthlyPolicy__8vr8mv3dl0t7lnvk37au9m3rdvlo0plmbgrdia1mq4ls67kpk58tmc44vjr5d7us/family1/ddfd0c29d15247d08eb9cb207da3d9eb 2023-03-05 19:56:13,683 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testMinorCompactionWithMonthlyPolicy__8vr8mv3dl0t7lnvk37au9m3rdvlo0plmbgrdia1mq4ls67kpk58tmc44vjr5d7us/family1/ddfd0c29d15247d08eb9cb207da3d9eb as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258/family1/db1a24c02cf2499ca176363d60559649_SeqId_49_ 2023-03-05 19:56:13,683 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.HStore(849): Loaded HFile hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testMinorCompactionWithMonthlyPolicy__8vr8mv3dl0t7lnvk37au9m3rdvlo0plmbgrdia1mq4ls67kpk58tmc44vjr5d7us/family1/ddfd0c29d15247d08eb9cb207da3d9eb into 014dc240b491a64066ab7b01d2b21258/family1 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258/family1/db1a24c02cf2499ca176363d60559649_SeqId_49_ - updating store file list. 2023-03-05 19:56:13,687 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.HStore(882): Loaded HFile hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258/family1/db1a24c02cf2499ca176363d60559649_SeqId_49_ into 014dc240b491a64066ab7b01d2b21258/family1 2023-03-05 19:56:13,687 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.HStore(855): Successfully loaded hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testMinorCompactionWithMonthlyPolicy__8vr8mv3dl0t7lnvk37au9m3rdvlo0plmbgrdia1mq4ls67kpk58tmc44vjr5d7us/family1/ddfd0c29d15247d08eb9cb207da3d9eb into 014dc240b491a64066ab7b01d2b21258/family1 (new location: hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258/family1/db1a24c02cf2499ca176363d60559649_SeqId_49_) 2023-03-05 19:56:13,688 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testMinorCompactionWithMonthlyPolicy__8vr8mv3dl0t7lnvk37au9m3rdvlo0plmbgrdia1mq4ls67kpk58tmc44vjr5d7us/family1/ddfd0c29d15247d08eb9cb207da3d9eb 2023-03-05 19:56:13,688 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-03-05 19:56:13,689 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 352581 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-03-05 19:56:13,689 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.HStore(1912): 014dc240b491a64066ab7b01d2b21258/family1 is initiating minor compaction (all files) 2023-03-05 19:56:13,689 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@dc3ef4d; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-05 19:56:13,690 INFO [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 014dc240b491a64066ab7b01d2b21258/family1 in testMinorCompactionWithMonthlyPolicy,,1678046161366.014dc240b491a64066ab7b01d2b21258. 2023-03-05 19:56:13,690 INFO [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258/family1/1588d6f89ab744748ad1303bac273ab7, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258/family1/be7b46be5d474303a5262cc09dd652dd, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258/family1/e50a05d3a69e41b09fc18d18b943c8ea_SeqId_46_, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258/family1/db1a24c02cf2499ca176363d60559649_SeqId_49_] into tmpdir=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258/.tmp, totalSize=344.3 K 2023-03-05 19:56:13,690 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.Compactor(207): Compacting 1588d6f89ab744748ad1303bac273ab7, keycount=1000, bloomtype=ROW, size=166.0 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878843274 2023-03-05 19:56:13,691 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.Compactor(207): Compacting be7b46be5d474303a5262cc09dd652dd, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621243274 2023-03-05 19:56:13,691 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.Compactor(207): Compacting e50a05d3a69e41b09fc18d18b943c8ea_SeqId_46_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=47, earliestPutTs=1448965243274 2023-03-05 19:56:13,692 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] compactions.Compactor(207): Compacting db1a24c02cf2499ca176363d60559649_SeqId_49_, keycount=1001, bloomtype=ROW, size=166.2 K, encoding=NONE, compression=NONE, seqNum=50, earliestPutTs=1448014843274 2023-03-05 19:56:13,692 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-03-05 19:56:13,693 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-05 19:56:13,694 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151120711d81b6e315496c9a26b35a4e882239 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151120711d81b6e315496c9a26b35a4e882239 2023-03-05 19:56:13,695 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151128a57a746ca2f14c14be3286cb3e8aa12f to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151128a57a746ca2f14c14be3286cb3e8aa12f 2023-03-05 19:56:13,696 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201511308d5bfa94d68c4299b78097fa2f750041 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201511308d5bfa94d68c4299b78097fa2f750041 2023-03-05 19:56:13,696 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithMonthlyPolicy 3 expired mob files are deleted 2023-03-05 19:56:13,696 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-03-05 19:56:13,697 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-03-05 19:56:13,698 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-03-05 19:56:13,698 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-03-05 19:56:13,698 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-03-05 19:56:13,698 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] mob.MobUtils(840): end MOB compaction of files for table='testMinorCompactionWithMonthlyPolicy', column='family1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-05 19:56:13,701 DEBUG [PEWorker-1] locking.LockProcedure(242): UNLOCKED pid=115, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-03-05 19:56:13,702 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=115, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE in 1.8830 sec 2023-03-05 19:56:13,714 INFO [RS:0;jenkins-hbase4:37037-shortCompactions-0] throttle.PressureAwareThroughputController(145): 014dc240b491a64066ab7b01d2b21258#family1#compaction#166 average throughput is 14.15 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-05 19:56:13,960 INFO [Listener at localhost/40787] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMinorCompactionWithMonthlyPolicy Thread=255 (was 253) - Thread LEAK? -, OpenFileDescriptor=640 (was 639) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=83 (was 86), ProcessCount=171 (was 171), AvailableMemoryMB=5266 (was 4502) - AvailableMemoryMB LEAK? - 2023-03-05 19:56:13,974 INFO [Listener at localhost/40787] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionFromAdmin Thread=255, OpenFileDescriptor=640, MaxFileDescriptor=60000, SystemLoadAverage=83, ProcessCount=171, AvailableMemoryMB=5265 2023-03-05 19:56:14,003 DEBUG [Listener at localhost/40787] crypto.Encryption(565): Installed org.apache.hadoop.hbase.io.crypto.KeyProviderForTesting into key provider cache 2023-03-05 19:56:14,084 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMajorCompactionFromAdmin', {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10\xCB\xA0\xE4\xC9z\xCF\xFF+\x1FU\xE1|\xD4\xB2#\x14"\x10F0\xDA\xDEX\xBEDK,1\xF2\xE1\x11\x8C\x8F\xEE*\x10V\xB4q\x5CP\x03\x05\x1F/\xCA\x00\x5C{c\x8Bh2\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-03-05 19:56:14,085 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure2.ProcedureExecutor(1029): Stored pid=116, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionFromAdmin 2023-03-05 19:56:14,087 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_PRE_OPERATION 2023-03-05 19:56:14,087 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionFromAdmin" procId is: 116 2023-03-05 19:56:14,088 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-05 19:56:14,088 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-03-05 19:56:14,090 DEBUG [HFileArchiver-20] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/.tmp/data/default/testMajorCompactionFromAdmin/484f8ac3504f985fdafb904759fb56ee 2023-03-05 19:56:14,090 DEBUG [HFileArchiver-21] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/.tmp/data/default/testMajorCompactionFromAdmin/806fed504069a9293ba794165a04fd64 2023-03-05 19:56:14,090 DEBUG [HFileArchiver-22] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/.tmp/data/default/testMajorCompactionFromAdmin/27d678a97a95df40c894e3fe4540a1f2 2023-03-05 19:56:14,090 DEBUG [HFileArchiver-20] backup.HFileArchiver(153): Directory hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/.tmp/data/default/testMajorCompactionFromAdmin/484f8ac3504f985fdafb904759fb56ee empty. 2023-03-05 19:56:14,090 DEBUG [HFileArchiver-21] backup.HFileArchiver(153): Directory hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/.tmp/data/default/testMajorCompactionFromAdmin/806fed504069a9293ba794165a04fd64 empty. 2023-03-05 19:56:14,090 DEBUG [HFileArchiver-22] backup.HFileArchiver(153): Directory hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/.tmp/data/default/testMajorCompactionFromAdmin/27d678a97a95df40c894e3fe4540a1f2 empty. 2023-03-05 19:56:14,091 DEBUG [HFileArchiver-20] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/.tmp/data/default/testMajorCompactionFromAdmin/484f8ac3504f985fdafb904759fb56ee 2023-03-05 19:56:14,091 DEBUG [HFileArchiver-21] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/.tmp/data/default/testMajorCompactionFromAdmin/806fed504069a9293ba794165a04fd64 2023-03-05 19:56:14,091 DEBUG [HFileArchiver-22] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/.tmp/data/default/testMajorCompactionFromAdmin/27d678a97a95df40c894e3fe4540a1f2 2023-03-05 19:56:14,091 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived testMajorCompactionFromAdmin regions 2023-03-05 19:56:14,099 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/.tmp/data/default/testMajorCompactionFromAdmin/.tabledesc/.tableinfo.0000000001 2023-03-05 19:56:14,100 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(7675): creating {ENCODED => 484f8ac3504f985fdafb904759fb56ee, NAME => 'testMajorCompactionFromAdmin,,1678046174038.484f8ac3504f985fdafb904759fb56ee.', STARTKEY => '', ENDKEY => '1'}, tableDescriptor='testMajorCompactionFromAdmin', {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10\xCB\xA0\xE4\xC9z\xCF\xFF+\x1FU\xE1|\xD4\xB2#\x14"\x10F0\xDA\xDEX\xBEDK,1\xF2\xE1\x11\x8C\x8F\xEE*\x10V\xB4q\x5CP\x03\x05\x1F/\xCA\x00\x5C{c\x8Bh2\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/.tmp 2023-03-05 19:56:14,100 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(7675): creating {ENCODED => 806fed504069a9293ba794165a04fd64, NAME => 'testMajorCompactionFromAdmin,1,1678046174038.806fed504069a9293ba794165a04fd64.', STARTKEY => '1', ENDKEY => '2'}, tableDescriptor='testMajorCompactionFromAdmin', {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10\xCB\xA0\xE4\xC9z\xCF\xFF+\x1FU\xE1|\xD4\xB2#\x14"\x10F0\xDA\xDEX\xBEDK,1\xF2\xE1\x11\x8C\x8F\xEE*\x10V\xB4q\x5CP\x03\x05\x1F/\xCA\x00\x5C{c\x8Bh2\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/.tmp 2023-03-05 19:56:14,104 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(7675): creating {ENCODED => 27d678a97a95df40c894e3fe4540a1f2, NAME => 'testMajorCompactionFromAdmin,2,1678046174038.27d678a97a95df40c894e3fe4540a1f2.', STARTKEY => '2', ENDKEY => ''}, tableDescriptor='testMajorCompactionFromAdmin', {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10\xCB\xA0\xE4\xC9z\xCF\xFF+\x1FU\xE1|\xD4\xB2#\x14"\x10F0\xDA\xDEX\xBEDK,1\xF2\xE1\x11\x8C\x8F\xEE*\x10V\xB4q\x5CP\x03\x05\x1F/\xCA\x00\x5C{c\x8Bh2\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/.tmp 2023-03-05 19:56:14,117 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(866): Instantiated testMajorCompactionFromAdmin,,1678046174038.484f8ac3504f985fdafb904759fb56ee.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-05 19:56:14,117 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1604): Closing 484f8ac3504f985fdafb904759fb56ee, disabling compactions & flushes 2023-03-05 19:56:14,117 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1626): Closing region testMajorCompactionFromAdmin,,1678046174038.484f8ac3504f985fdafb904759fb56ee. 2023-03-05 19:56:14,118 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionFromAdmin,,1678046174038.484f8ac3504f985fdafb904759fb56ee. 2023-03-05 19:56:14,118 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionFromAdmin,,1678046174038.484f8ac3504f985fdafb904759fb56ee. after waiting 0 ms 2023-03-05 19:56:14,118 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionFromAdmin,,1678046174038.484f8ac3504f985fdafb904759fb56ee. 2023-03-05 19:56:14,118 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1838): Closed testMajorCompactionFromAdmin,,1678046174038.484f8ac3504f985fdafb904759fb56ee. 2023-03-05 19:56:14,118 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1558): Region close journal for 484f8ac3504f985fdafb904759fb56ee: 2023-03-05 19:56:14,120 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(866): Instantiated testMajorCompactionFromAdmin,1,1678046174038.806fed504069a9293ba794165a04fd64.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-05 19:56:14,120 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1604): Closing 806fed504069a9293ba794165a04fd64, disabling compactions & flushes 2023-03-05 19:56:14,120 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1626): Closing region testMajorCompactionFromAdmin,1,1678046174038.806fed504069a9293ba794165a04fd64. 2023-03-05 19:56:14,120 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionFromAdmin,1,1678046174038.806fed504069a9293ba794165a04fd64. 2023-03-05 19:56:14,120 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionFromAdmin,1,1678046174038.806fed504069a9293ba794165a04fd64. after waiting 0 ms 2023-03-05 19:56:14,120 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionFromAdmin,1,1678046174038.806fed504069a9293ba794165a04fd64. 2023-03-05 19:56:14,120 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1838): Closed testMajorCompactionFromAdmin,1,1678046174038.806fed504069a9293ba794165a04fd64. 2023-03-05 19:56:14,120 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1558): Region close journal for 806fed504069a9293ba794165a04fd64: 2023-03-05 19:56:14,124 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(866): Instantiated testMajorCompactionFromAdmin,2,1678046174038.27d678a97a95df40c894e3fe4540a1f2.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-05 19:56:14,124 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1604): Closing 27d678a97a95df40c894e3fe4540a1f2, disabling compactions & flushes 2023-03-05 19:56:14,124 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1626): Closing region testMajorCompactionFromAdmin,2,1678046174038.27d678a97a95df40c894e3fe4540a1f2. 2023-03-05 19:56:14,124 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionFromAdmin,2,1678046174038.27d678a97a95df40c894e3fe4540a1f2. 2023-03-05 19:56:14,124 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionFromAdmin,2,1678046174038.27d678a97a95df40c894e3fe4540a1f2. after waiting 0 ms 2023-03-05 19:56:14,124 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionFromAdmin,2,1678046174038.27d678a97a95df40c894e3fe4540a1f2. 2023-03-05 19:56:14,124 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1838): Closed testMajorCompactionFromAdmin,2,1678046174038.27d678a97a95df40c894e3fe4540a1f2. 2023-03-05 19:56:14,124 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1558): Region close journal for 27d678a97a95df40c894e3fe4540a1f2: 2023-03-05 19:56:14,126 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_ADD_TO_META 2023-03-05 19:56:14,127 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionFromAdmin,,1678046174038.484f8ac3504f985fdafb904759fb56ee.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1678046174127"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678046174127"}]},"ts":"1678046174127"} 2023-03-05 19:56:14,127 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionFromAdmin,1,1678046174038.806fed504069a9293ba794165a04fd64.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1678046174127"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678046174127"}]},"ts":"1678046174127"} 2023-03-05 19:56:14,127 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionFromAdmin,2,1678046174038.27d678a97a95df40c894e3fe4540a1f2.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1678046174127"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678046174127"}]},"ts":"1678046174127"} 2023-03-05 19:56:14,129 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-03-05 19:56:14,130 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-05 19:56:14,130 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionFromAdmin","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678046174130"}]},"ts":"1678046174130"} 2023-03-05 19:56:14,131 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionFromAdmin, state=ENABLING in hbase:meta 2023-03-05 19:56:14,134 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=117, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=484f8ac3504f985fdafb904759fb56ee, ASSIGN}, {pid=118, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=806fed504069a9293ba794165a04fd64, ASSIGN}, {pid=119, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=27d678a97a95df40c894e3fe4540a1f2, ASSIGN}] 2023-03-05 19:56:14,136 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=119, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=27d678a97a95df40c894e3fe4540a1f2, ASSIGN 2023-03-05 19:56:14,136 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=117, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=484f8ac3504f985fdafb904759fb56ee, ASSIGN 2023-03-05 19:56:14,137 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=118, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=806fed504069a9293ba794165a04fd64, ASSIGN 2023-03-05 19:56:14,137 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=119, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=27d678a97a95df40c894e3fe4540a1f2, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,37037,1678046018983; forceNewPlan=false, retain=false 2023-03-05 19:56:14,137 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=117, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=484f8ac3504f985fdafb904759fb56ee, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,37037,1678046018983; forceNewPlan=false, retain=false 2023-03-05 19:56:14,137 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=118, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=806fed504069a9293ba794165a04fd64, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,37037,1678046018983; forceNewPlan=false, retain=false 2023-03-05 19:56:14,164 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258/.tmp/family1/479d5becea5043529156171b0e9f1792 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258/family1/479d5becea5043529156171b0e9f1792 2023-03-05 19:56:14,168 INFO [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 014dc240b491a64066ab7b01d2b21258/family1 of 014dc240b491a64066ab7b01d2b21258 into 479d5becea5043529156171b0e9f1792(size=167.4 K), total size for store is 167.4 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-05 19:56:14,168 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 014dc240b491a64066ab7b01d2b21258: 2023-03-05 19:56:14,169 INFO [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMinorCompactionWithMonthlyPolicy,,1678046161366.014dc240b491a64066ab7b01d2b21258., storeName=family1, fileCount=4, fileSize=344.3 K (166.0 K, 6.6 K, 5.5 K, 166.2 K), priority=2, time=1678046173689; duration=0sec 2023-03-05 19:56:14,169 DEBUG [RS:0;jenkins-hbase4:37037-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-05 19:56:14,185 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258/family1/1588d6f89ab744748ad1303bac273ab7, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258/family1/be7b46be5d474303a5262cc09dd652dd, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258/family1/e50a05d3a69e41b09fc18d18b943c8ea_SeqId_46_, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258/family1/db1a24c02cf2499ca176363d60559649_SeqId_49_] to archive 2023-03-05 19:56:14,185 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-05 19:56:14,187 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258/family1/1588d6f89ab744748ad1303bac273ab7 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258/family1/1588d6f89ab744748ad1303bac273ab7 2023-03-05 19:56:14,188 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258/family1/be7b46be5d474303a5262cc09dd652dd to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258/family1/be7b46be5d474303a5262cc09dd652dd 2023-03-05 19:56:14,189 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-03-05 19:56:14,189 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258/family1/e50a05d3a69e41b09fc18d18b943c8ea_SeqId_46_ to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258/family1/e50a05d3a69e41b09fc18d18b943c8ea_SeqId_46_ 2023-03-05 19:56:14,190 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258/family1/db1a24c02cf2499ca176363d60559649_SeqId_49_ to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258/family1/db1a24c02cf2499ca176363d60559649_SeqId_49_ 2023-03-05 19:56:14,289 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=118 updating hbase:meta row=806fed504069a9293ba794165a04fd64, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:14,289 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=117 updating hbase:meta row=484f8ac3504f985fdafb904759fb56ee, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:14,290 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionFromAdmin,1,1678046174038.806fed504069a9293ba794165a04fd64.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1678046174289"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678046174289"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678046174289"}]},"ts":"1678046174289"} 2023-03-05 19:56:14,289 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=119 updating hbase:meta row=27d678a97a95df40c894e3fe4540a1f2, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:14,290 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionFromAdmin,,1678046174038.484f8ac3504f985fdafb904759fb56ee.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1678046174289"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678046174289"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678046174289"}]},"ts":"1678046174289"} 2023-03-05 19:56:14,290 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionFromAdmin,2,1678046174038.27d678a97a95df40c894e3fe4540a1f2.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1678046174289"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678046174289"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678046174289"}]},"ts":"1678046174289"} 2023-03-05 19:56:14,291 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=120, ppid=118, state=RUNNABLE; OpenRegionProcedure 806fed504069a9293ba794165a04fd64, server=jenkins-hbase4.apache.org,37037,1678046018983}] 2023-03-05 19:56:14,292 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=121, ppid=117, state=RUNNABLE; OpenRegionProcedure 484f8ac3504f985fdafb904759fb56ee, server=jenkins-hbase4.apache.org,37037,1678046018983}] 2023-03-05 19:56:14,292 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=122, ppid=119, state=RUNNABLE; OpenRegionProcedure 27d678a97a95df40c894e3fe4540a1f2, server=jenkins-hbase4.apache.org,37037,1678046018983}] 2023-03-05 19:56:14,390 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-03-05 19:56:14,446 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionFromAdmin,1,1678046174038.806fed504069a9293ba794165a04fd64. 2023-03-05 19:56:14,446 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 806fed504069a9293ba794165a04fd64, NAME => 'testMajorCompactionFromAdmin,1,1678046174038.806fed504069a9293ba794165a04fd64.', STARTKEY => '1', ENDKEY => '2'} 2023-03-05 19:56:14,446 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionFromAdmin 806fed504069a9293ba794165a04fd64 2023-03-05 19:56:14,446 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionFromAdmin,1,1678046174038.806fed504069a9293ba794165a04fd64.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-05 19:56:14,447 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 806fed504069a9293ba794165a04fd64 2023-03-05 19:56:14,447 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 806fed504069a9293ba794165a04fd64 2023-03-05 19:56:14,474 INFO [StoreOpener-806fed504069a9293ba794165a04fd64-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 806fed504069a9293ba794165a04fd64 2023-03-05 19:56:14,476 DEBUG [StoreOpener-806fed504069a9293ba794165a04fd64-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/806fed504069a9293ba794165a04fd64/family1 2023-03-05 19:56:14,476 DEBUG [StoreOpener-806fed504069a9293ba794165a04fd64-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/806fed504069a9293ba794165a04fd64/family1 2023-03-05 19:56:14,476 INFO [StoreOpener-806fed504069a9293ba794165a04fd64-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 806fed504069a9293ba794165a04fd64 columnFamilyName family1 2023-03-05 19:56:14,478 INFO [StoreOpener-806fed504069a9293ba794165a04fd64-1] regionserver.HStore(310): Store=806fed504069a9293ba794165a04fd64/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-05 19:56:14,478 INFO [StoreOpener-806fed504069a9293ba794165a04fd64-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 806fed504069a9293ba794165a04fd64 2023-03-05 19:56:14,479 DEBUG [StoreOpener-806fed504069a9293ba794165a04fd64-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/806fed504069a9293ba794165a04fd64/family2 2023-03-05 19:56:14,479 DEBUG [StoreOpener-806fed504069a9293ba794165a04fd64-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/806fed504069a9293ba794165a04fd64/family2 2023-03-05 19:56:14,479 INFO [StoreOpener-806fed504069a9293ba794165a04fd64-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 806fed504069a9293ba794165a04fd64 columnFamilyName family2 2023-03-05 19:56:14,480 INFO [StoreOpener-806fed504069a9293ba794165a04fd64-1] regionserver.HStore(310): Store=806fed504069a9293ba794165a04fd64/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-05 19:56:14,481 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/806fed504069a9293ba794165a04fd64 2023-03-05 19:56:14,481 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/806fed504069a9293ba794165a04fd64 2023-03-05 19:56:14,483 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testMajorCompactionFromAdmin descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-05 19:56:14,483 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 806fed504069a9293ba794165a04fd64 2023-03-05 19:56:14,489 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/806fed504069a9293ba794165a04fd64/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-05 19:56:14,489 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 806fed504069a9293ba794165a04fd64; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68334462, jitterRate=0.0182628333568573}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-05 19:56:14,489 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 806fed504069a9293ba794165a04fd64: 2023-03-05 19:56:14,490 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionFromAdmin,1,1678046174038.806fed504069a9293ba794165a04fd64., pid=120, masterSystemTime=1678046174443 2023-03-05 19:56:14,491 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionFromAdmin,1,1678046174038.806fed504069a9293ba794165a04fd64. 2023-03-05 19:56:14,491 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionFromAdmin,1,1678046174038.806fed504069a9293ba794165a04fd64. 2023-03-05 19:56:14,491 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionFromAdmin,,1678046174038.484f8ac3504f985fdafb904759fb56ee. 2023-03-05 19:56:14,491 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 484f8ac3504f985fdafb904759fb56ee, NAME => 'testMajorCompactionFromAdmin,,1678046174038.484f8ac3504f985fdafb904759fb56ee.', STARTKEY => '', ENDKEY => '1'} 2023-03-05 19:56:14,491 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=118 updating hbase:meta row=806fed504069a9293ba794165a04fd64, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:14,491 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionFromAdmin 484f8ac3504f985fdafb904759fb56ee 2023-03-05 19:56:14,491 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionFromAdmin,1,1678046174038.806fed504069a9293ba794165a04fd64.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1678046174491"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678046174491"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678046174491"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678046174491"}]},"ts":"1678046174491"} 2023-03-05 19:56:14,491 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionFromAdmin,,1678046174038.484f8ac3504f985fdafb904759fb56ee.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-05 19:56:14,491 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 484f8ac3504f985fdafb904759fb56ee 2023-03-05 19:56:14,492 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 484f8ac3504f985fdafb904759fb56ee 2023-03-05 19:56:14,496 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=120, resume processing ppid=118 2023-03-05 19:56:14,496 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=120, ppid=118, state=SUCCESS; OpenRegionProcedure 806fed504069a9293ba794165a04fd64, server=jenkins-hbase4.apache.org,37037,1678046018983 in 203 msec 2023-03-05 19:56:14,497 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=118, ppid=116, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=806fed504069a9293ba794165a04fd64, ASSIGN in 362 msec 2023-03-05 19:56:14,520 INFO [StoreOpener-484f8ac3504f985fdafb904759fb56ee-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 484f8ac3504f985fdafb904759fb56ee 2023-03-05 19:56:14,522 DEBUG [StoreOpener-484f8ac3504f985fdafb904759fb56ee-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/484f8ac3504f985fdafb904759fb56ee/family1 2023-03-05 19:56:14,522 DEBUG [StoreOpener-484f8ac3504f985fdafb904759fb56ee-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/484f8ac3504f985fdafb904759fb56ee/family1 2023-03-05 19:56:14,522 INFO [StoreOpener-484f8ac3504f985fdafb904759fb56ee-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 484f8ac3504f985fdafb904759fb56ee columnFamilyName family1 2023-03-05 19:56:14,523 INFO [StoreOpener-484f8ac3504f985fdafb904759fb56ee-1] regionserver.HStore(310): Store=484f8ac3504f985fdafb904759fb56ee/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-05 19:56:14,523 INFO [StoreOpener-484f8ac3504f985fdafb904759fb56ee-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 484f8ac3504f985fdafb904759fb56ee 2023-03-05 19:56:14,525 DEBUG [StoreOpener-484f8ac3504f985fdafb904759fb56ee-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/484f8ac3504f985fdafb904759fb56ee/family2 2023-03-05 19:56:14,525 DEBUG [StoreOpener-484f8ac3504f985fdafb904759fb56ee-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/484f8ac3504f985fdafb904759fb56ee/family2 2023-03-05 19:56:14,525 INFO [StoreOpener-484f8ac3504f985fdafb904759fb56ee-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 484f8ac3504f985fdafb904759fb56ee columnFamilyName family2 2023-03-05 19:56:14,526 INFO [StoreOpener-484f8ac3504f985fdafb904759fb56ee-1] regionserver.HStore(310): Store=484f8ac3504f985fdafb904759fb56ee/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-05 19:56:14,526 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/484f8ac3504f985fdafb904759fb56ee 2023-03-05 19:56:14,527 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/484f8ac3504f985fdafb904759fb56ee 2023-03-05 19:56:14,529 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testMajorCompactionFromAdmin descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-05 19:56:14,530 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 484f8ac3504f985fdafb904759fb56ee 2023-03-05 19:56:14,531 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/484f8ac3504f985fdafb904759fb56ee/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-05 19:56:14,532 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 484f8ac3504f985fdafb904759fb56ee; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69412926, jitterRate=0.03433319926261902}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-05 19:56:14,532 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 484f8ac3504f985fdafb904759fb56ee: 2023-03-05 19:56:14,533 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionFromAdmin,,1678046174038.484f8ac3504f985fdafb904759fb56ee., pid=121, masterSystemTime=1678046174443 2023-03-05 19:56:14,534 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionFromAdmin,,1678046174038.484f8ac3504f985fdafb904759fb56ee. 2023-03-05 19:56:14,534 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionFromAdmin,,1678046174038.484f8ac3504f985fdafb904759fb56ee. 2023-03-05 19:56:14,534 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionFromAdmin,2,1678046174038.27d678a97a95df40c894e3fe4540a1f2. 2023-03-05 19:56:14,535 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 27d678a97a95df40c894e3fe4540a1f2, NAME => 'testMajorCompactionFromAdmin,2,1678046174038.27d678a97a95df40c894e3fe4540a1f2.', STARTKEY => '2', ENDKEY => ''} 2023-03-05 19:56:14,535 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=117 updating hbase:meta row=484f8ac3504f985fdafb904759fb56ee, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:14,535 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionFromAdmin 27d678a97a95df40c894e3fe4540a1f2 2023-03-05 19:56:14,535 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionFromAdmin,2,1678046174038.27d678a97a95df40c894e3fe4540a1f2.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-05 19:56:14,535 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionFromAdmin,,1678046174038.484f8ac3504f985fdafb904759fb56ee.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1678046174535"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678046174535"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678046174535"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678046174535"}]},"ts":"1678046174535"} 2023-03-05 19:56:14,535 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 27d678a97a95df40c894e3fe4540a1f2 2023-03-05 19:56:14,535 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 27d678a97a95df40c894e3fe4540a1f2 2023-03-05 19:56:14,538 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=121, resume processing ppid=117 2023-03-05 19:56:14,538 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=121, ppid=117, state=SUCCESS; OpenRegionProcedure 484f8ac3504f985fdafb904759fb56ee, server=jenkins-hbase4.apache.org,37037,1678046018983 in 244 msec 2023-03-05 19:56:14,539 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=117, ppid=116, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=484f8ac3504f985fdafb904759fb56ee, ASSIGN in 404 msec 2023-03-05 19:56:14,577 INFO [StoreOpener-27d678a97a95df40c894e3fe4540a1f2-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 27d678a97a95df40c894e3fe4540a1f2 2023-03-05 19:56:14,578 DEBUG [StoreOpener-27d678a97a95df40c894e3fe4540a1f2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/27d678a97a95df40c894e3fe4540a1f2/family1 2023-03-05 19:56:14,579 DEBUG [StoreOpener-27d678a97a95df40c894e3fe4540a1f2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/27d678a97a95df40c894e3fe4540a1f2/family1 2023-03-05 19:56:14,579 INFO [StoreOpener-27d678a97a95df40c894e3fe4540a1f2-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 27d678a97a95df40c894e3fe4540a1f2 columnFamilyName family1 2023-03-05 19:56:14,580 INFO [StoreOpener-27d678a97a95df40c894e3fe4540a1f2-1] regionserver.HStore(310): Store=27d678a97a95df40c894e3fe4540a1f2/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-05 19:56:14,580 INFO [StoreOpener-27d678a97a95df40c894e3fe4540a1f2-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 27d678a97a95df40c894e3fe4540a1f2 2023-03-05 19:56:14,581 DEBUG [StoreOpener-27d678a97a95df40c894e3fe4540a1f2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/27d678a97a95df40c894e3fe4540a1f2/family2 2023-03-05 19:56:14,581 DEBUG [StoreOpener-27d678a97a95df40c894e3fe4540a1f2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/27d678a97a95df40c894e3fe4540a1f2/family2 2023-03-05 19:56:14,582 INFO [StoreOpener-27d678a97a95df40c894e3fe4540a1f2-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 27d678a97a95df40c894e3fe4540a1f2 columnFamilyName family2 2023-03-05 19:56:14,582 INFO [StoreOpener-27d678a97a95df40c894e3fe4540a1f2-1] regionserver.HStore(310): Store=27d678a97a95df40c894e3fe4540a1f2/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-05 19:56:14,583 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/27d678a97a95df40c894e3fe4540a1f2 2023-03-05 19:56:14,583 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/27d678a97a95df40c894e3fe4540a1f2 2023-03-05 19:56:14,585 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testMajorCompactionFromAdmin descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-05 19:56:14,585 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 27d678a97a95df40c894e3fe4540a1f2 2023-03-05 19:56:14,587 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/27d678a97a95df40c894e3fe4540a1f2/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-05 19:56:14,587 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 27d678a97a95df40c894e3fe4540a1f2; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59834084, jitterRate=-0.10840266942977905}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-05 19:56:14,587 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 27d678a97a95df40c894e3fe4540a1f2: 2023-03-05 19:56:14,588 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionFromAdmin,2,1678046174038.27d678a97a95df40c894e3fe4540a1f2., pid=122, masterSystemTime=1678046174443 2023-03-05 19:56:14,590 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionFromAdmin,2,1678046174038.27d678a97a95df40c894e3fe4540a1f2. 2023-03-05 19:56:14,590 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionFromAdmin,2,1678046174038.27d678a97a95df40c894e3fe4540a1f2. 2023-03-05 19:56:14,590 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=119 updating hbase:meta row=27d678a97a95df40c894e3fe4540a1f2, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:14,590 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionFromAdmin,2,1678046174038.27d678a97a95df40c894e3fe4540a1f2.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1678046174590"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678046174590"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678046174590"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678046174590"}]},"ts":"1678046174590"} 2023-03-05 19:56:14,593 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=122, resume processing ppid=119 2023-03-05 19:56:14,593 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=122, ppid=119, state=SUCCESS; OpenRegionProcedure 27d678a97a95df40c894e3fe4540a1f2, server=jenkins-hbase4.apache.org,37037,1678046018983 in 300 msec 2023-03-05 19:56:14,594 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=119, resume processing ppid=116 2023-03-05 19:56:14,594 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=119, ppid=116, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=27d678a97a95df40c894e3fe4540a1f2, ASSIGN in 459 msec 2023-03-05 19:56:14,595 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-05 19:56:14,595 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionFromAdmin","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678046174595"}]},"ts":"1678046174595"} 2023-03-05 19:56:14,596 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionFromAdmin, state=ENABLED in hbase:meta 2023-03-05 19:56:14,598 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_POST_OPERATION 2023-03-05 19:56:14,599 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=116, state=SUCCESS; CreateTableProcedure table=testMajorCompactionFromAdmin in 514 msec 2023-03-05 19:56:14,691 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-03-05 19:56:14,691 INFO [Listener at localhost/40787] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionFromAdmin, procId: 116 completed 2023-03-05 19:56:14,699 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,,1678046174038.484f8ac3504f985fdafb904759fb56ee. with WAL disabled. Data may be lost in the event of a crash. 2023-03-05 19:56:14,700 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,1,1678046174038.806fed504069a9293ba794165a04fd64. with WAL disabled. Data may be lost in the event of a crash. 2023-03-05 19:56:14,700 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,2,1678046174038.27d678a97a95df40c894e3fe4540a1f2. with WAL disabled. Data may be lost in the event of a crash. 2023-03-05 19:56:14,703 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-05 19:56:14,705 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-03-05 19:56:14,705 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-03-05 19:56:14,705 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:56:14,705 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-03-05 19:56:14,705 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-03-05 19:56:14,705 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-05 19:56:14,705 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-05 19:56:14,710 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:56:14,711 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:14,711 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:56:14,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:56:14,711 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:14,711 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-05 19:56:14,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-05 19:56:14,711 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-05 19:56:14,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-05 19:56:14,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-05 19:56:14,712 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-03-05 19:56:14,712 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-03-05 19:56:14,712 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-03-05 19:56:14,712 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:56:14,712 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-03-05 19:56:14,712 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1678046174038.484f8ac3504f985fdafb904759fb56ee. 2023-03-05 19:56:14,713 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1678046174038.484f8ac3504f985fdafb904759fb56ee. started... 2023-03-05 19:56:14,714 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1678046174038.27d678a97a95df40c894e3fe4540a1f2. 2023-03-05 19:56:14,714 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1678046174038.27d678a97a95df40c894e3fe4540a1f2. started... 2023-03-05 19:56:14,714 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 484f8ac3504f985fdafb904759fb56ee 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-03-05 19:56:14,721 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-05 19:56:14,721 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-05 19:56:14,722 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 27d678a97a95df40c894e3fe4540a1f2 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-03-05 19:56:14,722 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1678046174038.806fed504069a9293ba794165a04fd64. 2023-03-05 19:56:14,723 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1678046174038.806fed504069a9293ba794165a04fd64. started... 2023-03-05 19:56:14,723 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 806fed504069a9293ba794165a04fd64 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-03-05 19:56:15,199 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230305dc111309c35549b582ac70838b1fc273 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230305dc111309c35549b582ac70838b1fc273 2023-03-05 19:56:15,199 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230305eef0ba6b6aa5445c8043212cee716589 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230305eef0ba6b6aa5445c8043212cee716589 2023-03-05 19:56:15,218 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230305e19b92a441f24b5c9b7841418f7111b5 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230305e19b92a441f24b5c9b7841418f7111b5 2023-03-05 19:56:15,241 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/806fed504069a9293ba794165a04fd64/.tmp/family1/76f1d4eac62f464dafe23cb47b0fa058 2023-03-05 19:56:15,626 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/27d678a97a95df40c894e3fe4540a1f2/.tmp/family1/b95506c2d50a46bf86830e6f67dda314 2023-03-05 19:56:15,637 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/484f8ac3504f985fdafb904759fb56ee/.tmp/family1/a8ca0cdd660d49d999a663d2a800caad 2023-03-05 19:56:15,677 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230305d23a39c2c83a48eabbe60761f0f33350 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b20230305d23a39c2c83a48eabbe60761f0f33350 2023-03-05 19:56:15,682 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/806fed504069a9293ba794165a04fd64/.tmp/family2/d850c9c1ee784e71a5fbb3fc2927b071 2023-03-05 19:56:15,684 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303058178340807ed4e599d14f55424938c10 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e202303058178340807ed4e599d14f55424938c10 2023-03-05 19:56:15,687 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/806fed504069a9293ba794165a04fd64/.tmp/family1/76f1d4eac62f464dafe23cb47b0fa058 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/806fed504069a9293ba794165a04fd64/family1/76f1d4eac62f464dafe23cb47b0fa058 2023-03-05 19:56:15,695 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/484f8ac3504f985fdafb904759fb56ee/.tmp/family2/3031bd61f0e5467a907106f8f0333070 2023-03-05 19:56:15,700 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/484f8ac3504f985fdafb904759fb56ee/.tmp/family1/a8ca0cdd660d49d999a663d2a800caad as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/484f8ac3504f985fdafb904759fb56ee/family1/a8ca0cdd660d49d999a663d2a800caad 2023-03-05 19:56:15,735 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/806fed504069a9293ba794165a04fd64/family1/76f1d4eac62f464dafe23cb47b0fa058, entries=4, sequenceid=5, filesize=5.5 K 2023-03-05 19:56:15,736 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/806fed504069a9293ba794165a04fd64/.tmp/family2/d850c9c1ee784e71a5fbb3fc2927b071 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/806fed504069a9293ba794165a04fd64/family2/d850c9c1ee784e71a5fbb3fc2927b071 2023-03-05 19:56:15,737 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/484f8ac3504f985fdafb904759fb56ee/family1/a8ca0cdd660d49d999a663d2a800caad, entries=4, sequenceid=5, filesize=5.5 K 2023-03-05 19:56:15,739 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/484f8ac3504f985fdafb904759fb56ee/.tmp/family2/3031bd61f0e5467a907106f8f0333070 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/484f8ac3504f985fdafb904759fb56ee/family2/3031bd61f0e5467a907106f8f0333070 2023-03-05 19:56:15,742 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/806fed504069a9293ba794165a04fd64/family2/d850c9c1ee784e71a5fbb3fc2927b071, entries=2, sequenceid=5, filesize=5.1 K 2023-03-05 19:56:15,743 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/484f8ac3504f985fdafb904759fb56ee/family2/3031bd61f0e5467a907106f8f0333070, entries=2, sequenceid=5, filesize=5.1 K 2023-03-05 19:56:15,743 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 806fed504069a9293ba794165a04fd64 in 1020ms, sequenceid=5, compaction requested=false 2023-03-05 19:56:15,744 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionFromAdmin' 2023-03-05 19:56:15,744 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 484f8ac3504f985fdafb904759fb56ee in 1030ms, sequenceid=5, compaction requested=false 2023-03-05 19:56:15,744 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionFromAdmin' 2023-03-05 19:56:15,749 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 806fed504069a9293ba794165a04fd64: 2023-03-05 19:56:15,749 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1678046174038.806fed504069a9293ba794165a04fd64. 2023-03-05 19:56:15,749 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-05 19:56:15,751 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 484f8ac3504f985fdafb904759fb56ee: 2023-03-05 19:56:15,751 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1678046174038.484f8ac3504f985fdafb904759fb56ee. 2023-03-05 19:56:15,751 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-05 19:56:16,073 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230305308a3b2f96464e46be8b17e57fe7e462 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c20230305308a3b2f96464e46be8b17e57fe7e462 2023-03-05 19:56:16,478 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/27d678a97a95df40c894e3fe4540a1f2/.tmp/family2/52ea01697ce04731a85b6040cee8e0d2 2023-03-05 19:56:16,482 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/27d678a97a95df40c894e3fe4540a1f2/.tmp/family1/b95506c2d50a46bf86830e6f67dda314 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/27d678a97a95df40c894e3fe4540a1f2/family1/b95506c2d50a46bf86830e6f67dda314 2023-03-05 19:56:16,505 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/27d678a97a95df40c894e3fe4540a1f2/family1/b95506c2d50a46bf86830e6f67dda314, entries=4, sequenceid=5, filesize=5.5 K 2023-03-05 19:56:16,506 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/27d678a97a95df40c894e3fe4540a1f2/.tmp/family2/52ea01697ce04731a85b6040cee8e0d2 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/27d678a97a95df40c894e3fe4540a1f2/family2/52ea01697ce04731a85b6040cee8e0d2 2023-03-05 19:56:16,509 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/27d678a97a95df40c894e3fe4540a1f2/family2/52ea01697ce04731a85b6040cee8e0d2, entries=2, sequenceid=5, filesize=5.1 K 2023-03-05 19:56:16,510 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 27d678a97a95df40c894e3fe4540a1f2 in 1788ms, sequenceid=5, compaction requested=false 2023-03-05 19:56:16,510 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 27d678a97a95df40c894e3fe4540a1f2: 2023-03-05 19:56:16,510 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1678046174038.27d678a97a95df40c894e3fe4540a1f2. 2023-03-05 19:56:16,510 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-05 19:56:16,510 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-05 19:56:16,510 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:16,510 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-03-05 19:56:16,510 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-03-05 19:56:16,512 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-05 19:56:16,512 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:16,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:16,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:56:16,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:56:16,512 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-05 19:56:16,512 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-05 19:56:16,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:56:16,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:56:16,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-05 19:56:16,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:16,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:56:16,513 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-03-05 19:56:16,513 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@12220b95[Count = 0] remaining members to acquire global barrier 2023-03-05 19:56:16,513 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-03-05 19:56:16,513 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-05 19:56:16,514 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-05 19:56:16,514 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-05 19:56:16,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-05 19:56:16,515 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-03-05 19:56:16,515 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-03-05 19:56:16,515 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,37037,1678046018983' in zk 2023-03-05 19:56:16,515 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:16,515 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-05 19:56:16,516 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-03-05 19:56:16,516 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:16,516 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:56:16,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:16,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:56:16,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:56:16,516 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-03-05 19:56:16,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:56:16,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:56:16,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-05 19:56:16,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:16,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:56:16,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-05 19:56:16,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:16,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase4.apache.org,37037,1678046018983': 2023-03-05 19:56:16,518 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-03-05 19:56:16,518 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-03-05 19:56:16,518 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-05 19:56:16,518 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-05 19:56:16,518 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-03-05 19:56:16,518 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-05 19:56:16,521 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-05 19:56:16,521 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-05 19:56:16,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-05 19:56:16,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:56:16,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:56:16,521 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-05 19:56:16,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-05 19:56:16,521 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:56:16,521 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:16,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:56:16,521 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:56:16,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:56:16,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-05 19:56:16,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-05 19:56:16,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:56:16,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-05 19:56:16,523 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:16,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:16,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:56:16,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-05 19:56:16,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:16,525 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:16,525 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-05 19:56:16,525 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:56:16,525 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:56:16,525 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin because node does not exist (not an error) 2023-03-05 19:56:16,525 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-05 19:56:16,525 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:56:16,525 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-03-05 19:56:16,525 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-05 19:56:16,525 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:56:16,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:56:16,525 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:16,525 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-05 19:56:16,525 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-05 19:56:16,525 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-05 19:56:16,525 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-03-05 19:56:16,525 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-05 19:56:16,525 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:56:16,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:56:16,626 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-05 19:56:16,626 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-05 19:56:16,627 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,,1678046174038.484f8ac3504f985fdafb904759fb56ee. with WAL disabled. Data may be lost in the event of a crash. 2023-03-05 19:56:16,627 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,1,1678046174038.806fed504069a9293ba794165a04fd64. with WAL disabled. Data may be lost in the event of a crash. 2023-03-05 19:56:16,628 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,2,1678046174038.27d678a97a95df40c894e3fe4540a1f2. with WAL disabled. Data may be lost in the event of a crash. 2023-03-05 19:56:16,630 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-05 19:56:16,631 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionFromAdmin was in running list but was completed. Accepting new attempt. 2023-03-05 19:56:16,631 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-03-05 19:56:16,631 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-03-05 19:56:16,631 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:56:16,631 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-03-05 19:56:16,631 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-03-05 19:56:16,632 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-05 19:56:16,632 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-05 19:56:16,633 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:16,633 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:56:16,633 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:56:16,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:56:16,634 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:16,634 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-05 19:56:16,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-05 19:56:16,634 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-05 19:56:16,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-05 19:56:16,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-05 19:56:16,634 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-03-05 19:56:16,635 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionFromAdmin is still present, removing 2023-03-05 19:56:16,635 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-03-05 19:56:16,635 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-03-05 19:56:16,635 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:56:16,635 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-03-05 19:56:16,635 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1678046174038.484f8ac3504f985fdafb904759fb56ee. 2023-03-05 19:56:16,635 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-05 19:56:16,635 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-05 19:56:16,635 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1678046174038.806fed504069a9293ba794165a04fd64. 2023-03-05 19:56:16,635 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1678046174038.27d678a97a95df40c894e3fe4540a1f2. 2023-03-05 19:56:16,635 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1678046174038.27d678a97a95df40c894e3fe4540a1f2. started... 2023-03-05 19:56:16,635 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1678046174038.484f8ac3504f985fdafb904759fb56ee. started... 2023-03-05 19:56:16,635 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1678046174038.806fed504069a9293ba794165a04fd64. started... 2023-03-05 19:56:16,635 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 27d678a97a95df40c894e3fe4540a1f2 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-03-05 19:56:16,635 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 806fed504069a9293ba794165a04fd64 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-03-05 19:56:16,635 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 484f8ac3504f985fdafb904759fb56ee 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-03-05 19:56:17,086 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023030547b5e4a18bdb437b99f5f85dbe33db3f to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e2023030547b5e4a18bdb437b99f5f85dbe33db3f 2023-03-05 19:56:17,097 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303054dc539352e8843ce948b0ea4502741a4 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202303054dc539352e8843ce948b0ea4502741a4 2023-03-05 19:56:17,097 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230305aa7bac24fda0413e83c0cfb39758a1ac to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230305aa7bac24fda0413e83c0cfb39758a1ac 2023-03-05 19:56:17,501 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/484f8ac3504f985fdafb904759fb56ee/.tmp/family1/c30e2cd1b2074a5c88ad6306a5de034b 2023-03-05 19:56:17,512 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/806fed504069a9293ba794165a04fd64/.tmp/family1/f048033a76f842c2ab3d1f55862f4751 2023-03-05 19:56:17,512 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/27d678a97a95df40c894e3fe4540a1f2/.tmp/family1/3b82c3fc0bfa4990b0c9e011cd31d35a 2023-03-05 19:56:17,546 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303053850b2326f3643db8a1704792b7fe141 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e202303053850b2326f3643db8a1704792b7fe141 2023-03-05 19:56:17,560 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/484f8ac3504f985fdafb904759fb56ee/.tmp/family2/52a339f9dbe24d748095616145b6069c 2023-03-05 19:56:17,564 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/484f8ac3504f985fdafb904759fb56ee/.tmp/family1/c30e2cd1b2074a5c88ad6306a5de034b as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/484f8ac3504f985fdafb904759fb56ee/family1/c30e2cd1b2074a5c88ad6306a5de034b 2023-03-05 19:56:17,565 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023030546357a64ec3b4bfaa06ae941a1810e21 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b2023030546357a64ec3b4bfaa06ae941a1810e21 2023-03-05 19:56:17,590 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/484f8ac3504f985fdafb904759fb56ee/family1/c30e2cd1b2074a5c88ad6306a5de034b, entries=4, sequenceid=9, filesize=5.5 K 2023-03-05 19:56:17,591 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/484f8ac3504f985fdafb904759fb56ee/.tmp/family2/52a339f9dbe24d748095616145b6069c as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/484f8ac3504f985fdafb904759fb56ee/family2/52a339f9dbe24d748095616145b6069c 2023-03-05 19:56:17,594 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/484f8ac3504f985fdafb904759fb56ee/family2/52a339f9dbe24d748095616145b6069c, entries=2, sequenceid=9, filesize=5.1 K 2023-03-05 19:56:17,595 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 484f8ac3504f985fdafb904759fb56ee in 960ms, sequenceid=9, compaction requested=false 2023-03-05 19:56:17,595 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 484f8ac3504f985fdafb904759fb56ee: 2023-03-05 19:56:17,595 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1678046174038.484f8ac3504f985fdafb904759fb56ee. 2023-03-05 19:56:17,595 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-05 19:56:17,961 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023030559bd8de049244c2abf8bea4ec85a4f6d to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c2023030559bd8de049244c2abf8bea4ec85a4f6d 2023-03-05 19:56:17,977 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/806fed504069a9293ba794165a04fd64/.tmp/family2/fb21e0ddcdfe49a188a2f30bd3074b5e 2023-03-05 19:56:17,980 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/806fed504069a9293ba794165a04fd64/.tmp/family1/f048033a76f842c2ab3d1f55862f4751 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/806fed504069a9293ba794165a04fd64/family1/f048033a76f842c2ab3d1f55862f4751 2023-03-05 19:56:18,003 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/806fed504069a9293ba794165a04fd64/family1/f048033a76f842c2ab3d1f55862f4751, entries=4, sequenceid=9, filesize=5.5 K 2023-03-05 19:56:18,004 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/806fed504069a9293ba794165a04fd64/.tmp/family2/fb21e0ddcdfe49a188a2f30bd3074b5e as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/806fed504069a9293ba794165a04fd64/family2/fb21e0ddcdfe49a188a2f30bd3074b5e 2023-03-05 19:56:18,007 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/806fed504069a9293ba794165a04fd64/family2/fb21e0ddcdfe49a188a2f30bd3074b5e, entries=2, sequenceid=9, filesize=5.1 K 2023-03-05 19:56:18,007 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 806fed504069a9293ba794165a04fd64 in 1372ms, sequenceid=9, compaction requested=false 2023-03-05 19:56:18,007 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 806fed504069a9293ba794165a04fd64: 2023-03-05 19:56:18,007 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1678046174038.806fed504069a9293ba794165a04fd64. 2023-03-05 19:56:18,007 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-05 19:56:18,365 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/27d678a97a95df40c894e3fe4540a1f2/.tmp/family2/a51cee78384b4559ae358ceb3fc3a35a 2023-03-05 19:56:18,369 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/27d678a97a95df40c894e3fe4540a1f2/.tmp/family1/3b82c3fc0bfa4990b0c9e011cd31d35a as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/27d678a97a95df40c894e3fe4540a1f2/family1/3b82c3fc0bfa4990b0c9e011cd31d35a 2023-03-05 19:56:18,391 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/27d678a97a95df40c894e3fe4540a1f2/family1/3b82c3fc0bfa4990b0c9e011cd31d35a, entries=4, sequenceid=9, filesize=5.5 K 2023-03-05 19:56:18,392 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/27d678a97a95df40c894e3fe4540a1f2/.tmp/family2/a51cee78384b4559ae358ceb3fc3a35a as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/27d678a97a95df40c894e3fe4540a1f2/family2/a51cee78384b4559ae358ceb3fc3a35a 2023-03-05 19:56:18,395 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/27d678a97a95df40c894e3fe4540a1f2/family2/a51cee78384b4559ae358ceb3fc3a35a, entries=2, sequenceid=9, filesize=5.1 K 2023-03-05 19:56:18,395 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 27d678a97a95df40c894e3fe4540a1f2 in 1760ms, sequenceid=9, compaction requested=false 2023-03-05 19:56:18,395 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 27d678a97a95df40c894e3fe4540a1f2: 2023-03-05 19:56:18,396 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1678046174038.27d678a97a95df40c894e3fe4540a1f2. 2023-03-05 19:56:18,396 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-05 19:56:18,396 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-05 19:56:18,396 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:18,396 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-03-05 19:56:18,396 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-03-05 19:56:18,397 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:18,397 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-05 19:56:18,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:18,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:56:18,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:56:18,398 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-05 19:56:18,398 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-05 19:56:18,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:56:18,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:56:18,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-05 19:56:18,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:18,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:56:18,399 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-03-05 19:56:18,399 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@431aad35[Count = 0] remaining members to acquire global barrier 2023-03-05 19:56:18,399 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-03-05 19:56:18,399 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-05 19:56:18,400 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-05 19:56:18,400 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-05 19:56:18,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-05 19:56:18,400 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-03-05 19:56:18,400 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-03-05 19:56:18,400 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,37037,1678046018983' in zk 2023-03-05 19:56:18,400 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:18,400 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-05 19:56:18,401 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:18,401 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-03-05 19:56:18,401 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:56:18,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:18,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:56:18,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:56:18,401 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-03-05 19:56:18,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:56:18,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:56:18,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-05 19:56:18,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:18,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:56:18,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-05 19:56:18,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:18,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase4.apache.org,37037,1678046018983': 2023-03-05 19:56:18,403 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-03-05 19:56:18,403 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-03-05 19:56:18,403 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-05 19:56:18,403 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-05 19:56:18,404 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-03-05 19:56:18,404 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-05 19:56:18,406 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-05 19:56:18,406 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-05 19:56:18,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-05 19:56:18,406 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-05 19:56:18,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-05 19:56:18,406 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:56:18,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:56:18,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:56:18,406 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:56:18,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:56:18,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:56:18,406 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:18,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-05 19:56:18,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-05 19:56:18,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:56:18,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-05 19:56:18,407 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:18,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:18,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:56:18,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-05 19:56:18,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:18,409 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:56:18,409 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:18,409 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:56:18,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:56:18,409 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:56:18,409 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin because node does not exist (not an error) 2023-03-05 19:56:18,409 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-03-05 19:56:18,410 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-05 19:56:18,409 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:56:18,409 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-05 19:56:18,410 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:56:18,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:56:18,410 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-05 19:56:18,410 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:18,410 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-03-05 19:56:18,410 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-05 19:56:18,410 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-05 19:56:18,410 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-05 19:56:18,410 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-05 19:56:18,510 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-05 19:56:18,511 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-05 19:56:18,512 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,,1678046174038.484f8ac3504f985fdafb904759fb56ee. with WAL disabled. Data may be lost in the event of a crash. 2023-03-05 19:56:18,512 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,1,1678046174038.806fed504069a9293ba794165a04fd64. with WAL disabled. Data may be lost in the event of a crash. 2023-03-05 19:56:18,512 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,2,1678046174038.27d678a97a95df40c894e3fe4540a1f2. with WAL disabled. Data may be lost in the event of a crash. 2023-03-05 19:56:18,514 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-05 19:56:18,516 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionFromAdmin was in running list but was completed. Accepting new attempt. 2023-03-05 19:56:18,516 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-03-05 19:56:18,516 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-03-05 19:56:18,516 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:56:18,516 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-03-05 19:56:18,516 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-03-05 19:56:18,517 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-05 19:56:18,517 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-05 19:56:18,518 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:56:18,518 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:18,518 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:56:18,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:56:18,518 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:18,518 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-05 19:56:18,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-05 19:56:18,519 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-05 19:56:18,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-05 19:56:18,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-05 19:56:18,519 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-03-05 19:56:18,519 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionFromAdmin is still present, removing 2023-03-05 19:56:18,519 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-03-05 19:56:18,519 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-03-05 19:56:18,519 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:56:18,519 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-03-05 19:56:18,520 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1678046174038.484f8ac3504f985fdafb904759fb56ee. 2023-03-05 19:56:18,520 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1678046174038.484f8ac3504f985fdafb904759fb56ee. started... 2023-03-05 19:56:18,520 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-05 19:56:18,520 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-05 19:56:18,520 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1678046174038.806fed504069a9293ba794165a04fd64. 2023-03-05 19:56:18,520 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1678046174038.27d678a97a95df40c894e3fe4540a1f2. 2023-03-05 19:56:18,520 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1678046174038.806fed504069a9293ba794165a04fd64. started... 2023-03-05 19:56:18,520 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 484f8ac3504f985fdafb904759fb56ee 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-03-05 19:56:18,520 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 806fed504069a9293ba794165a04fd64 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-03-05 19:56:18,520 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1678046174038.27d678a97a95df40c894e3fe4540a1f2. started... 2023-03-05 19:56:18,520 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 27d678a97a95df40c894e3fe4540a1f2 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-03-05 19:56:18,568 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230305b11ae57e62354bb09d12e0a7a06e4000 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230305b11ae57e62354bb09d12e0a7a06e4000 2023-03-05 19:56:18,572 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303056f8ec686125e46d6840dd7a563cc76ea to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202303056f8ec686125e46d6840dd7a563cc76ea 2023-03-05 19:56:18,588 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/484f8ac3504f985fdafb904759fb56ee/.tmp/family1/a60d9d6fe7dc4dadb27b1be63f37ce9b 2023-03-05 19:56:18,590 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/27d678a97a95df40c894e3fe4540a1f2/.tmp/family1/2ee46606b01b42ebafce97b03f725567 2023-03-05 19:56:18,621 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303057a5f01b02a59442eb541b3732295ca2e to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c202303057a5f01b02a59442eb541b3732295ca2e 2023-03-05 19:56:18,625 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/27d678a97a95df40c894e3fe4540a1f2/.tmp/family2/0fe6bf4b584a4bdcaaa0d012a328c8d8 2023-03-05 19:56:18,629 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/27d678a97a95df40c894e3fe4540a1f2/.tmp/family1/2ee46606b01b42ebafce97b03f725567 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/27d678a97a95df40c894e3fe4540a1f2/family1/2ee46606b01b42ebafce97b03f725567 2023-03-05 19:56:18,631 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023030536e0553b57884c3d9636ad8d0320d360 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e2023030536e0553b57884c3d9636ad8d0320d360 2023-03-05 19:56:18,643 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/484f8ac3504f985fdafb904759fb56ee/.tmp/family2/3f2cab9d93f24668a5948bdf58aae419 2023-03-05 19:56:18,647 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/484f8ac3504f985fdafb904759fb56ee/.tmp/family1/a60d9d6fe7dc4dadb27b1be63f37ce9b as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/484f8ac3504f985fdafb904759fb56ee/family1/a60d9d6fe7dc4dadb27b1be63f37ce9b 2023-03-05 19:56:18,664 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/27d678a97a95df40c894e3fe4540a1f2/family1/2ee46606b01b42ebafce97b03f725567, entries=4, sequenceid=13, filesize=5.5 K 2023-03-05 19:56:18,664 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/27d678a97a95df40c894e3fe4540a1f2/.tmp/family2/0fe6bf4b584a4bdcaaa0d012a328c8d8 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/27d678a97a95df40c894e3fe4540a1f2/family2/0fe6bf4b584a4bdcaaa0d012a328c8d8 2023-03-05 19:56:18,668 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/27d678a97a95df40c894e3fe4540a1f2/family2/0fe6bf4b584a4bdcaaa0d012a328c8d8, entries=2, sequenceid=13, filesize=5.1 K 2023-03-05 19:56:18,669 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 27d678a97a95df40c894e3fe4540a1f2 in 149ms, sequenceid=13, compaction requested=true 2023-03-05 19:56:18,669 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 27d678a97a95df40c894e3fe4540a1f2: 2023-03-05 19:56:18,669 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1678046174038.27d678a97a95df40c894e3fe4540a1f2. 2023-03-05 19:56:18,669 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-05 19:56:18,674 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/484f8ac3504f985fdafb904759fb56ee/family1/a60d9d6fe7dc4dadb27b1be63f37ce9b, entries=4, sequenceid=13, filesize=5.5 K 2023-03-05 19:56:18,674 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/484f8ac3504f985fdafb904759fb56ee/.tmp/family2/3f2cab9d93f24668a5948bdf58aae419 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/484f8ac3504f985fdafb904759fb56ee/family2/3f2cab9d93f24668a5948bdf58aae419 2023-03-05 19:56:18,677 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/484f8ac3504f985fdafb904759fb56ee/family2/3f2cab9d93f24668a5948bdf58aae419, entries=2, sequenceid=13, filesize=5.1 K 2023-03-05 19:56:18,678 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 484f8ac3504f985fdafb904759fb56ee in 158ms, sequenceid=13, compaction requested=true 2023-03-05 19:56:18,678 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 484f8ac3504f985fdafb904759fb56ee: 2023-03-05 19:56:18,678 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1678046174038.484f8ac3504f985fdafb904759fb56ee. 2023-03-05 19:56:18,678 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-05 19:56:18,969 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023030553bf5dc2a7a64ce59d0701205796ffeb to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b2023030553bf5dc2a7a64ce59d0701205796ffeb 2023-03-05 19:56:19,383 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/806fed504069a9293ba794165a04fd64/.tmp/family1/297eb1800270488ea5afc665defca8d0 2023-03-05 19:56:19,815 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230305cc393e4b43fa43cc999199ba0ec50406 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b20230305cc393e4b43fa43cc999199ba0ec50406 2023-03-05 19:56:19,989 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-05 19:56:20,219 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/806fed504069a9293ba794165a04fd64/.tmp/family2/8f022f0990784143bd64e4161d6ca99d 2023-03-05 19:56:20,223 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/806fed504069a9293ba794165a04fd64/.tmp/family1/297eb1800270488ea5afc665defca8d0 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/806fed504069a9293ba794165a04fd64/family1/297eb1800270488ea5afc665defca8d0 2023-03-05 19:56:20,246 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/806fed504069a9293ba794165a04fd64/family1/297eb1800270488ea5afc665defca8d0, entries=4, sequenceid=13, filesize=5.5 K 2023-03-05 19:56:20,247 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/806fed504069a9293ba794165a04fd64/.tmp/family2/8f022f0990784143bd64e4161d6ca99d as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/806fed504069a9293ba794165a04fd64/family2/8f022f0990784143bd64e4161d6ca99d 2023-03-05 19:56:20,250 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/806fed504069a9293ba794165a04fd64/family2/8f022f0990784143bd64e4161d6ca99d, entries=2, sequenceid=13, filesize=5.1 K 2023-03-05 19:56:20,251 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 806fed504069a9293ba794165a04fd64 in 1731ms, sequenceid=13, compaction requested=true 2023-03-05 19:56:20,251 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 806fed504069a9293ba794165a04fd64: 2023-03-05 19:56:20,251 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1678046174038.806fed504069a9293ba794165a04fd64. 2023-03-05 19:56:20,251 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-05 19:56:20,251 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-05 19:56:20,251 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:20,251 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-03-05 19:56:20,251 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-03-05 19:56:20,253 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:20,253 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-05 19:56:20,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:20,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:56:20,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:56:20,253 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-05 19:56:20,253 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-05 19:56:20,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:56:20,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:56:20,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-05 19:56:20,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:20,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:56:20,254 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-03-05 19:56:20,254 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5afa2bea[Count = 0] remaining members to acquire global barrier 2023-03-05 19:56:20,254 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-03-05 19:56:20,254 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-05 19:56:20,256 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-05 19:56:20,256 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-05 19:56:20,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-05 19:56:20,256 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-03-05 19:56:20,256 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-03-05 19:56:20,256 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,37037,1678046018983' in zk 2023-03-05 19:56:20,256 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:20,256 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-05 19:56:20,257 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-03-05 19:56:20,257 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:20,257 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:56:20,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:20,257 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-03-05 19:56:20,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:56:20,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:56:20,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:56:20,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:56:20,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-05 19:56:20,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:20,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:56:20,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-05 19:56:20,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:20,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase4.apache.org,37037,1678046018983': 2023-03-05 19:56:20,260 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-03-05 19:56:20,260 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-05 19:56:20,260 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-05 19:56:20,260 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-03-05 19:56:20,260 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-03-05 19:56:20,260 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-05 19:56:20,261 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-05 19:56:20,261 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-05 19:56:20,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-05 19:56:20,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:56:20,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:56:20,261 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-05 19:56:20,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-05 19:56:20,261 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:56:20,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:56:20,262 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:20,262 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:56:20,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:56:20,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-05 19:56:20,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-05 19:56:20,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:56:20,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-05 19:56:20,263 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:20,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:20,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:56:20,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-05 19:56:20,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:20,265 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:20,265 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-05 19:56:20,265 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-05 19:56:20,265 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:20,265 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-05 19:56:20,265 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-03-05 19:56:20,265 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-05 19:56:20,265 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin because node does not exist (not an error) 2023-03-05 19:56:20,265 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-05 19:56:20,265 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-05 19:56:20,265 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:56:20,265 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:56:20,265 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:56:20,265 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:56:20,265 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-03-05 19:56:20,265 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-05 19:56:20,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:56:20,265 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:56:20,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:56:20,365 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-05 19:56:20,366 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-05 19:56:20,367 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,,1678046174038.484f8ac3504f985fdafb904759fb56ee. with WAL disabled. Data may be lost in the event of a crash. 2023-03-05 19:56:20,367 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,1,1678046174038.806fed504069a9293ba794165a04fd64. with WAL disabled. Data may be lost in the event of a crash. 2023-03-05 19:56:20,367 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,2,1678046174038.27d678a97a95df40c894e3fe4540a1f2. with WAL disabled. Data may be lost in the event of a crash. 2023-03-05 19:56:20,369 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-05 19:56:20,371 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionFromAdmin was in running list but was completed. Accepting new attempt. 2023-03-05 19:56:20,371 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-03-05 19:56:20,371 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-03-05 19:56:20,371 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:56:20,371 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-03-05 19:56:20,371 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-03-05 19:56:20,371 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-05 19:56:20,371 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-05 19:56:20,377 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:20,377 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:56:20,377 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:56:20,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:56:20,377 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:20,377 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-05 19:56:20,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-05 19:56:20,378 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-05 19:56:20,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-05 19:56:20,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-05 19:56:20,378 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-03-05 19:56:20,378 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionFromAdmin is still present, removing 2023-03-05 19:56:20,378 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-03-05 19:56:20,378 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-03-05 19:56:20,378 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:56:20,378 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-03-05 19:56:20,379 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1678046174038.484f8ac3504f985fdafb904759fb56ee. 2023-03-05 19:56:20,379 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1678046174038.484f8ac3504f985fdafb904759fb56ee. started... 2023-03-05 19:56:20,379 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-05 19:56:20,379 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-05 19:56:20,379 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1678046174038.806fed504069a9293ba794165a04fd64. 2023-03-05 19:56:20,379 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1678046174038.27d678a97a95df40c894e3fe4540a1f2. 2023-03-05 19:56:20,379 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 484f8ac3504f985fdafb904759fb56ee 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-03-05 19:56:20,379 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1678046174038.27d678a97a95df40c894e3fe4540a1f2. started... 2023-03-05 19:56:20,379 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1678046174038.806fed504069a9293ba794165a04fd64. started... 2023-03-05 19:56:20,379 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 27d678a97a95df40c894e3fe4540a1f2 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-03-05 19:56:20,379 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 806fed504069a9293ba794165a04fd64 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-03-05 19:56:20,420 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303053ec254fceddf4af78c1846209703f42e to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202303053ec254fceddf4af78c1846209703f42e 2023-03-05 19:56:20,825 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230305588a97d843c94c12ae18c76a49eb3040 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230305588a97d843c94c12ae18c76a49eb3040 2023-03-05 19:56:20,825 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023030524b5b5e287994efc8204f27810fa8262 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e2023030524b5b5e287994efc8204f27810fa8262 2023-03-05 19:56:20,834 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/27d678a97a95df40c894e3fe4540a1f2/.tmp/family1/e7913b8bbfcb454e88855230488cbb3f 2023-03-05 19:56:21,246 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/806fed504069a9293ba794165a04fd64/.tmp/family1/d95d39f65ea04368a1051a20fc118ea8 2023-03-05 19:56:21,246 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/484f8ac3504f985fdafb904759fb56ee/.tmp/family1/e0a9f962fbdb4ad7829f27187915377a 2023-03-05 19:56:21,266 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230305984fca3b8beb42b8b944488acdf27c89 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c20230305984fca3b8beb42b8b944488acdf27c89 2023-03-05 19:56:21,670 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/27d678a97a95df40c894e3fe4540a1f2/.tmp/family2/0f0187e0342542c296b9416a59434cde 2023-03-05 19:56:21,674 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/27d678a97a95df40c894e3fe4540a1f2/.tmp/family1/e7913b8bbfcb454e88855230488cbb3f as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/27d678a97a95df40c894e3fe4540a1f2/family1/e7913b8bbfcb454e88855230488cbb3f 2023-03-05 19:56:21,694 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230305ef6aca8aeddb457097870dc56ff6dffa to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b20230305ef6aca8aeddb457097870dc56ff6dffa 2023-03-05 19:56:21,694 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023030506f64d08c15d4550b9dff8c65853c2d8 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e2023030506f64d08c15d4550b9dff8c65853c2d8 2023-03-05 19:56:21,704 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/806fed504069a9293ba794165a04fd64/.tmp/family2/5eff69a5399a48188ba32d73e7d90955 2023-03-05 19:56:21,705 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/27d678a97a95df40c894e3fe4540a1f2/family1/e7913b8bbfcb454e88855230488cbb3f, entries=4, sequenceid=17, filesize=5.5 K 2023-03-05 19:56:21,705 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/27d678a97a95df40c894e3fe4540a1f2/.tmp/family2/0f0187e0342542c296b9416a59434cde as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/27d678a97a95df40c894e3fe4540a1f2/family2/0f0187e0342542c296b9416a59434cde 2023-03-05 19:56:21,708 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/806fed504069a9293ba794165a04fd64/.tmp/family1/d95d39f65ea04368a1051a20fc118ea8 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/806fed504069a9293ba794165a04fd64/family1/d95d39f65ea04368a1051a20fc118ea8 2023-03-05 19:56:21,709 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/27d678a97a95df40c894e3fe4540a1f2/family2/0f0187e0342542c296b9416a59434cde, entries=2, sequenceid=17, filesize=5.1 K 2023-03-05 19:56:21,710 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 27d678a97a95df40c894e3fe4540a1f2 in 1331ms, sequenceid=17, compaction requested=true 2023-03-05 19:56:21,710 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 27d678a97a95df40c894e3fe4540a1f2: 2023-03-05 19:56:21,710 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1678046174038.27d678a97a95df40c894e3fe4540a1f2. 2023-03-05 19:56:21,710 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-05 19:56:21,731 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/806fed504069a9293ba794165a04fd64/family1/d95d39f65ea04368a1051a20fc118ea8, entries=4, sequenceid=17, filesize=5.5 K 2023-03-05 19:56:21,732 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/806fed504069a9293ba794165a04fd64/.tmp/family2/5eff69a5399a48188ba32d73e7d90955 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/806fed504069a9293ba794165a04fd64/family2/5eff69a5399a48188ba32d73e7d90955 2023-03-05 19:56:21,735 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/806fed504069a9293ba794165a04fd64/family2/5eff69a5399a48188ba32d73e7d90955, entries=2, sequenceid=17, filesize=5.1 K 2023-03-05 19:56:21,735 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 806fed504069a9293ba794165a04fd64 in 1356ms, sequenceid=17, compaction requested=true 2023-03-05 19:56:21,736 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 806fed504069a9293ba794165a04fd64: 2023-03-05 19:56:21,736 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1678046174038.806fed504069a9293ba794165a04fd64. 2023-03-05 19:56:21,736 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-05 19:56:22,105 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/484f8ac3504f985fdafb904759fb56ee/.tmp/family2/2a2ffad3470e41ed9d75c5ec3c7d57e9 2023-03-05 19:56:22,109 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/484f8ac3504f985fdafb904759fb56ee/.tmp/family1/e0a9f962fbdb4ad7829f27187915377a as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/484f8ac3504f985fdafb904759fb56ee/family1/e0a9f962fbdb4ad7829f27187915377a 2023-03-05 19:56:22,141 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/484f8ac3504f985fdafb904759fb56ee/family1/e0a9f962fbdb4ad7829f27187915377a, entries=4, sequenceid=17, filesize=5.5 K 2023-03-05 19:56:22,142 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/484f8ac3504f985fdafb904759fb56ee/.tmp/family2/2a2ffad3470e41ed9d75c5ec3c7d57e9 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/484f8ac3504f985fdafb904759fb56ee/family2/2a2ffad3470e41ed9d75c5ec3c7d57e9 2023-03-05 19:56:22,145 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/484f8ac3504f985fdafb904759fb56ee/family2/2a2ffad3470e41ed9d75c5ec3c7d57e9, entries=2, sequenceid=17, filesize=5.1 K 2023-03-05 19:56:22,146 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 484f8ac3504f985fdafb904759fb56ee in 1767ms, sequenceid=17, compaction requested=true 2023-03-05 19:56:22,146 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 484f8ac3504f985fdafb904759fb56ee: 2023-03-05 19:56:22,146 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1678046174038.484f8ac3504f985fdafb904759fb56ee. 2023-03-05 19:56:22,146 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-05 19:56:22,146 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-05 19:56:22,146 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:22,146 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-03-05 19:56:22,146 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-03-05 19:56:22,147 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-05 19:56:22,147 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:22,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:22,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:56:22,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:56:22,148 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-05 19:56:22,148 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-05 19:56:22,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:56:22,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:56:22,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-05 19:56:22,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:22,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:56:22,150 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-03-05 19:56:22,150 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3dfabc41[Count = 0] remaining members to acquire global barrier 2023-03-05 19:56:22,150 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-03-05 19:56:22,150 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-05 19:56:22,151 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-05 19:56:22,151 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-05 19:56:22,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-05 19:56:22,151 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-03-05 19:56:22,151 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-03-05 19:56:22,151 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,37037,1678046018983' in zk 2023-03-05 19:56:22,151 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:22,151 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-05 19:56:22,152 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-03-05 19:56:22,152 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:22,152 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:56:22,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:22,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:56:22,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:56:22,152 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-03-05 19:56:22,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:56:22,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:56:22,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-05 19:56:22,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:22,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:56:22,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-05 19:56:22,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:22,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase4.apache.org,37037,1678046018983': 2023-03-05 19:56:22,154 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-03-05 19:56:22,154 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-03-05 19:56:22,154 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-05 19:56:22,154 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-05 19:56:22,154 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-03-05 19:56:22,154 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-05 19:56:22,156 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-05 19:56:22,156 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-05 19:56:22,156 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-05 19:56:22,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-05 19:56:22,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:56:22,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:56:22,156 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:56:22,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-05 19:56:22,157 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:22,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:56:22,157 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:56:22,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:56:22,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-05 19:56:22,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-05 19:56:22,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:56:22,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-05 19:56:22,158 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:22,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:22,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:56:22,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-05 19:56:22,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:22,160 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:22,160 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-05 19:56:22,160 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:56:22,160 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:56:22,160 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-03-05 19:56:22,160 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-05 19:56:22,160 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:56:22,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:56:22,160 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:56:22,160 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-05 19:56:22,160 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:22,160 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-05 19:56:22,160 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-05 19:56:22,160 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-03-05 19:56:22,161 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:56:22,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:56:22,160 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-05 19:56:22,161 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-05 19:56:22,261 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-05 19:56:22,262 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-05 19:56:22,269 DEBUG [Listener at localhost/40787] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-05 19:56:22,276 DEBUG [Listener at localhost/40787] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-05 19:56:22,284 DEBUG [Listener at localhost/40787] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-05 19:56:22,285 DEBUG [Listener at localhost/40787] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-05 19:56:22,370 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-05 19:56:22,372 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionFromAdmin was in running list but was completed. Accepting new attempt. 2023-03-05 19:56:22,372 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-03-05 19:56:22,372 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-03-05 19:56:22,372 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:56:22,372 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-03-05 19:56:22,372 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-03-05 19:56:22,372 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-05 19:56:22,372 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-05 19:56:22,373 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:56:22,373 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:22,373 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:56:22,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:56:22,374 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:22,374 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-05 19:56:22,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-05 19:56:22,374 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-05 19:56:22,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-05 19:56:22,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-05 19:56:22,375 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-03-05 19:56:22,375 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionFromAdmin is still present, removing 2023-03-05 19:56:22,375 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-03-05 19:56:22,375 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-03-05 19:56:22,375 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-05 19:56:22,375 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-03-05 19:56:22,375 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1678046174038.484f8ac3504f985fdafb904759fb56ee. 2023-03-05 19:56:22,375 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1678046174038.27d678a97a95df40c894e3fe4540a1f2. 2023-03-05 19:56:22,375 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1678046174038.484f8ac3504f985fdafb904759fb56ee. started... 2023-03-05 19:56:22,375 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1678046174038.27d678a97a95df40c894e3fe4540a1f2. started... 2023-03-05 19:56:22,376 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 484f8ac3504f985fdafb904759fb56ee 2/2 column families, dataSize=138 B heapSize=944 B 2023-03-05 19:56:22,376 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 27d678a97a95df40c894e3fe4540a1f2 2/2 column families, dataSize=138 B heapSize=944 B 2023-03-05 19:56:22,376 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-05 19:56:22,376 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-05 19:56:22,376 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1678046174038.806fed504069a9293ba794165a04fd64. 2023-03-05 19:56:22,376 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1678046174038.806fed504069a9293ba794165a04fd64. started... 2023-03-05 19:56:22,376 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 806fed504069a9293ba794165a04fd64 2/2 column families, dataSize=138 B heapSize=944 B 2023-03-05 19:56:22,820 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/806fed504069a9293ba794165a04fd64/.tmp/family1/113c92cea7d64dd5b203b7c519cfa62c 2023-03-05 19:56:22,844 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 113c92cea7d64dd5b203b7c519cfa62c 2023-03-05 19:56:23,227 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/484f8ac3504f985fdafb904759fb56ee/.tmp/family1/235af62437d94a4aac6c151a20a173b3 2023-03-05 19:56:23,233 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/27d678a97a95df40c894e3fe4540a1f2/.tmp/family1/c86374b33b5145b08312cf08ad34c330 2023-03-05 19:56:23,259 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 235af62437d94a4aac6c151a20a173b3 2023-03-05 19:56:23,260 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c86374b33b5145b08312cf08ad34c330 2023-03-05 19:56:23,277 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/484f8ac3504f985fdafb904759fb56ee/.tmp/family2/637e42afeeb84981af879d9b012e8dae 2023-03-05 19:56:23,280 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 637e42afeeb84981af879d9b012e8dae 2023-03-05 19:56:23,280 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/484f8ac3504f985fdafb904759fb56ee/.tmp/family1/235af62437d94a4aac6c151a20a173b3 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/484f8ac3504f985fdafb904759fb56ee/family1/235af62437d94a4aac6c151a20a173b3 2023-03-05 19:56:23,303 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 235af62437d94a4aac6c151a20a173b3 2023-03-05 19:56:23,303 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/484f8ac3504f985fdafb904759fb56ee/family1/235af62437d94a4aac6c151a20a173b3, entries=3, sequenceid=23, filesize=5.2 K 2023-03-05 19:56:23,303 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/484f8ac3504f985fdafb904759fb56ee/.tmp/family2/637e42afeeb84981af879d9b012e8dae as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/484f8ac3504f985fdafb904759fb56ee/family2/637e42afeeb84981af879d9b012e8dae 2023-03-05 19:56:23,306 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 637e42afeeb84981af879d9b012e8dae 2023-03-05 19:56:23,306 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/484f8ac3504f985fdafb904759fb56ee/family2/637e42afeeb84981af879d9b012e8dae, entries=1, sequenceid=23, filesize=5.0 K 2023-03-05 19:56:23,307 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 484f8ac3504f985fdafb904759fb56ee in 931ms, sequenceid=23, compaction requested=true 2023-03-05 19:56:23,307 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 484f8ac3504f985fdafb904759fb56ee: 2023-03-05 19:56:23,307 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1678046174038.484f8ac3504f985fdafb904759fb56ee. 2023-03-05 19:56:23,307 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-05 19:56:23,658 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/806fed504069a9293ba794165a04fd64/.tmp/family2/694c4424907e47d294f0564a7ba4db41 2023-03-05 19:56:23,661 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 694c4424907e47d294f0564a7ba4db41 2023-03-05 19:56:23,662 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/806fed504069a9293ba794165a04fd64/.tmp/family1/113c92cea7d64dd5b203b7c519cfa62c as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/806fed504069a9293ba794165a04fd64/family1/113c92cea7d64dd5b203b7c519cfa62c 2023-03-05 19:56:23,686 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 113c92cea7d64dd5b203b7c519cfa62c 2023-03-05 19:56:23,686 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/806fed504069a9293ba794165a04fd64/family1/113c92cea7d64dd5b203b7c519cfa62c, entries=3, sequenceid=23, filesize=5.2 K 2023-03-05 19:56:23,686 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/806fed504069a9293ba794165a04fd64/.tmp/family2/694c4424907e47d294f0564a7ba4db41 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/806fed504069a9293ba794165a04fd64/family2/694c4424907e47d294f0564a7ba4db41 2023-03-05 19:56:23,690 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 694c4424907e47d294f0564a7ba4db41 2023-03-05 19:56:23,690 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/806fed504069a9293ba794165a04fd64/family2/694c4424907e47d294f0564a7ba4db41, entries=1, sequenceid=23, filesize=5.0 K 2023-03-05 19:56:23,690 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 806fed504069a9293ba794165a04fd64 in 1314ms, sequenceid=23, compaction requested=true 2023-03-05 19:56:23,690 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 806fed504069a9293ba794165a04fd64: 2023-03-05 19:56:23,690 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1678046174038.806fed504069a9293ba794165a04fd64. 2023-03-05 19:56:23,690 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-05 19:56:24,080 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/27d678a97a95df40c894e3fe4540a1f2/.tmp/family2/90f910f718734a4ab1f6602bbdf7d60e 2023-03-05 19:56:24,083 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 90f910f718734a4ab1f6602bbdf7d60e 2023-03-05 19:56:24,084 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/27d678a97a95df40c894e3fe4540a1f2/.tmp/family1/c86374b33b5145b08312cf08ad34c330 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/27d678a97a95df40c894e3fe4540a1f2/family1/c86374b33b5145b08312cf08ad34c330 2023-03-05 19:56:24,106 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c86374b33b5145b08312cf08ad34c330 2023-03-05 19:56:24,106 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/27d678a97a95df40c894e3fe4540a1f2/family1/c86374b33b5145b08312cf08ad34c330, entries=3, sequenceid=23, filesize=5.2 K 2023-03-05 19:56:24,107 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/27d678a97a95df40c894e3fe4540a1f2/.tmp/family2/90f910f718734a4ab1f6602bbdf7d60e as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/27d678a97a95df40c894e3fe4540a1f2/family2/90f910f718734a4ab1f6602bbdf7d60e 2023-03-05 19:56:24,110 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 90f910f718734a4ab1f6602bbdf7d60e 2023-03-05 19:56:24,110 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/27d678a97a95df40c894e3fe4540a1f2/family2/90f910f718734a4ab1f6602bbdf7d60e, entries=1, sequenceid=23, filesize=5.0 K 2023-03-05 19:56:24,111 INFO [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 27d678a97a95df40c894e3fe4540a1f2 in 1735ms, sequenceid=23, compaction requested=true 2023-03-05 19:56:24,111 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 27d678a97a95df40c894e3fe4540a1f2: 2023-03-05 19:56:24,111 DEBUG [rs(jenkins-hbase4.apache.org,37037,1678046018983)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1678046174038.27d678a97a95df40c894e3fe4540a1f2. 2023-03-05 19:56:24,111 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-05 19:56:24,111 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-05 19:56:24,111 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:24,111 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-03-05 19:56:24,111 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-03-05 19:56:24,113 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:24,113 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-05 19:56:24,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:24,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:56:24,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:56:24,113 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-05 19:56:24,113 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-05 19:56:24,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:56:24,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:56:24,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-05 19:56:24,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:24,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:56:24,114 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37037,1678046018983' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-03-05 19:56:24,114 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3f8666c8[Count = 0] remaining members to acquire global barrier 2023-03-05 19:56:24,114 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-03-05 19:56:24,114 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-05 19:56:24,116 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-05 19:56:24,116 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-05 19:56:24,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-05 19:56:24,116 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-03-05 19:56:24,116 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-03-05 19:56:24,116 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,37037,1678046018983' in zk 2023-03-05 19:56:24,116 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:24,116 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-05 19:56:24,117 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-03-05 19:56:24,117 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:24,118 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:56:24,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:24,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:56:24,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:56:24,118 DEBUG [member: 'jenkins-hbase4.apache.org,37037,1678046018983' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-03-05 19:56:24,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:56:24,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:56:24,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-05 19:56:24,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:24,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:56:24,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-05 19:56:24,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:24,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase4.apache.org,37037,1678046018983': 2023-03-05 19:56:24,120 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37037,1678046018983' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-03-05 19:56:24,120 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-03-05 19:56:24,120 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-05 19:56:24,120 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-05 19:56:24,120 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-03-05 19:56:24,120 INFO [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-05 19:56:24,121 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-05 19:56:24,121 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-05 19:56:24,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-05 19:56:24,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-05 19:56:24,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-05 19:56:24,121 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-05 19:56:24,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-05 19:56:24,121 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:56:24,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-05 19:56:24,122 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:24,122 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:56:24,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:56:24,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-05 19:56:24,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-05 19:56:24,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-05 19:56:24,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-05 19:56:24,123 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:24,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:24,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-05 19:56:24,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-05 19:56:24,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:24,124 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:24,124 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-05 19:56:24,124 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-05 19:56:24,125 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-03-05 19:56:24,125 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-05 19:56:24,125 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin because node does not exist (not an error) 2023-03-05 19:56:24,125 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-05 19:56:24,125 DEBUG [(jenkins-hbase4.apache.org,38221,1678046017738)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-05 19:56:24,125 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-05 19:56:24,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-05 19:56:24,125 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-03-05 19:56:24,125 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-05 19:56:24,125 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-05 19:56:24,125 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:24,125 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-05 19:56:24,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-05 19:56:24,125 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-05 19:56:24,125 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-05 19:56:24,125 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-05 19:56:24,225 DEBUG [Listener at localhost/40787] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-05 19:56:24,226 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38221] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-05 19:56:24,227 DEBUG [Listener at localhost/40787] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-05 19:56:24,227 DEBUG [Listener at localhost/40787] regionserver.HStore(1912): 484f8ac3504f985fdafb904759fb56ee/family1 is initiating major compaction (all files) 2023-03-05 19:56:24,227 INFO [Listener at localhost/40787] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-05 19:56:24,227 INFO [Listener at localhost/40787] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-05 19:56:24,227 INFO [Listener at localhost/40787] regionserver.HRegion(2259): Starting compaction of 484f8ac3504f985fdafb904759fb56ee/family1 in testMajorCompactionFromAdmin,,1678046174038.484f8ac3504f985fdafb904759fb56ee. 2023-03-05 19:56:24,228 INFO [Listener at localhost/40787] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/484f8ac3504f985fdafb904759fb56ee/family1/a8ca0cdd660d49d999a663d2a800caad, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/484f8ac3504f985fdafb904759fb56ee/family1/c30e2cd1b2074a5c88ad6306a5de034b, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/484f8ac3504f985fdafb904759fb56ee/family1/a60d9d6fe7dc4dadb27b1be63f37ce9b, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/484f8ac3504f985fdafb904759fb56ee/family1/e0a9f962fbdb4ad7829f27187915377a, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/484f8ac3504f985fdafb904759fb56ee/family1/235af62437d94a4aac6c151a20a173b3] into tmpdir=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/484f8ac3504f985fdafb904759fb56ee/.tmp, totalSize=27.1 K 2023-03-05 19:56:24,228 DEBUG [Listener at localhost/40787] compactions.Compactor(207): Compacting a8ca0cdd660d49d999a663d2a800caad, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678046174699 2023-03-05 19:56:24,229 DEBUG [Listener at localhost/40787] compactions.Compactor(207): Compacting c30e2cd1b2074a5c88ad6306a5de034b, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678046176627 2023-03-05 19:56:24,229 DEBUG [Listener at localhost/40787] compactions.Compactor(207): Compacting a60d9d6fe7dc4dadb27b1be63f37ce9b, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678046178512 2023-03-05 19:56:24,229 DEBUG [Listener at localhost/40787] compactions.Compactor(207): Compacting e0a9f962fbdb4ad7829f27187915377a, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678046180367 2023-03-05 19:56:24,230 DEBUG [Listener at localhost/40787] compactions.Compactor(207): Compacting 235af62437d94a4aac6c151a20a173b3, keycount=3, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-05 19:56:24,239 INFO [Listener at localhost/40787] throttle.PressureAwareThroughputController(145): 484f8ac3504f985fdafb904759fb56ee#family1#compaction#197 average throughput is 0.57 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-05 19:56:24,694 INFO [Listener at localhost/40787] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20230305d507491929fc4e0da0c8abe9c0c57e6f_del 2023-03-05 19:56:24,694 INFO [Listener at localhost/40787] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230305d507491929fc4e0da0c8abe9c0c57e6f_del to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230305d507491929fc4e0da0c8abe9c0c57e6f_del 2023-03-05 19:56:25,546 INFO [Listener at localhost/40787] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5eb2304412e74c25959b06e41102c96a 2023-03-05 19:56:25,547 DEBUG [Listener at localhost/40787] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/484f8ac3504f985fdafb904759fb56ee/.tmp/family1/5eb2304412e74c25959b06e41102c96a as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/484f8ac3504f985fdafb904759fb56ee/family1/5eb2304412e74c25959b06e41102c96a 2023-03-05 19:56:25,569 INFO [Listener at localhost/40787] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5eb2304412e74c25959b06e41102c96a 2023-03-05 19:56:25,570 INFO [Listener at localhost/40787] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 484f8ac3504f985fdafb904759fb56ee/family1 of 484f8ac3504f985fdafb904759fb56ee into 5eb2304412e74c25959b06e41102c96a(size=7.0 K), total size for store is 7.0 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-05 19:56:25,570 DEBUG [Listener at localhost/40787] regionserver.HRegion(2289): Compaction status journal for 484f8ac3504f985fdafb904759fb56ee: 2023-03-05 19:56:25,570 DEBUG [Listener at localhost/40787] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-05 19:56:25,570 DEBUG [Listener at localhost/40787] regionserver.HStore(1912): 484f8ac3504f985fdafb904759fb56ee/family2 is initiating major compaction (all files) 2023-03-05 19:56:25,570 INFO [Listener at localhost/40787] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-05 19:56:25,570 INFO [Listener at localhost/40787] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-05 19:56:25,570 INFO [Listener at localhost/40787] regionserver.HRegion(2259): Starting compaction of 484f8ac3504f985fdafb904759fb56ee/family2 in testMajorCompactionFromAdmin,,1678046174038.484f8ac3504f985fdafb904759fb56ee. 2023-03-05 19:56:25,570 INFO [Listener at localhost/40787] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/484f8ac3504f985fdafb904759fb56ee/family2/3031bd61f0e5467a907106f8f0333070, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/484f8ac3504f985fdafb904759fb56ee/family2/52a339f9dbe24d748095616145b6069c, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/484f8ac3504f985fdafb904759fb56ee/family2/3f2cab9d93f24668a5948bdf58aae419, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/484f8ac3504f985fdafb904759fb56ee/family2/2a2ffad3470e41ed9d75c5ec3c7d57e9, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/484f8ac3504f985fdafb904759fb56ee/family2/637e42afeeb84981af879d9b012e8dae] into tmpdir=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/484f8ac3504f985fdafb904759fb56ee/.tmp, totalSize=25.4 K 2023-03-05 19:56:25,571 DEBUG [Listener at localhost/40787] compactions.Compactor(207): Compacting 3031bd61f0e5467a907106f8f0333070, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678046174699 2023-03-05 19:56:25,571 DEBUG [Listener at localhost/40787] compactions.Compactor(207): Compacting 52a339f9dbe24d748095616145b6069c, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678046176627 2023-03-05 19:56:25,572 DEBUG [Listener at localhost/40787] compactions.Compactor(207): Compacting 3f2cab9d93f24668a5948bdf58aae419, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678046178512 2023-03-05 19:56:25,572 DEBUG [Listener at localhost/40787] compactions.Compactor(207): Compacting 2a2ffad3470e41ed9d75c5ec3c7d57e9, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678046180367 2023-03-05 19:56:25,572 DEBUG [Listener at localhost/40787] compactions.Compactor(207): Compacting 637e42afeeb84981af879d9b012e8dae, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-05 19:56:25,580 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/484f8ac3504f985fdafb904759fb56ee/family1/a8ca0cdd660d49d999a663d2a800caad, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/484f8ac3504f985fdafb904759fb56ee/family1/c30e2cd1b2074a5c88ad6306a5de034b, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/484f8ac3504f985fdafb904759fb56ee/family1/a60d9d6fe7dc4dadb27b1be63f37ce9b, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/484f8ac3504f985fdafb904759fb56ee/family1/e0a9f962fbdb4ad7829f27187915377a, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/484f8ac3504f985fdafb904759fb56ee/family1/235af62437d94a4aac6c151a20a173b3] to archive 2023-03-05 19:56:25,580 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-05 19:56:25,581 INFO [Listener at localhost/40787] throttle.PressureAwareThroughputController(145): 484f8ac3504f985fdafb904759fb56ee#family2#compaction#198 average throughput is 0.53 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-05 19:56:25,582 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/484f8ac3504f985fdafb904759fb56ee/family1/a8ca0cdd660d49d999a663d2a800caad to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionFromAdmin/484f8ac3504f985fdafb904759fb56ee/family1/a8ca0cdd660d49d999a663d2a800caad 2023-03-05 19:56:25,583 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/484f8ac3504f985fdafb904759fb56ee/family1/c30e2cd1b2074a5c88ad6306a5de034b to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionFromAdmin/484f8ac3504f985fdafb904759fb56ee/family1/c30e2cd1b2074a5c88ad6306a5de034b 2023-03-05 19:56:25,584 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/484f8ac3504f985fdafb904759fb56ee/family1/a60d9d6fe7dc4dadb27b1be63f37ce9b to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionFromAdmin/484f8ac3504f985fdafb904759fb56ee/family1/a60d9d6fe7dc4dadb27b1be63f37ce9b 2023-03-05 19:56:25,585 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/484f8ac3504f985fdafb904759fb56ee/family1/e0a9f962fbdb4ad7829f27187915377a to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionFromAdmin/484f8ac3504f985fdafb904759fb56ee/family1/e0a9f962fbdb4ad7829f27187915377a 2023-03-05 19:56:25,586 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/484f8ac3504f985fdafb904759fb56ee/family1/235af62437d94a4aac6c151a20a173b3 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionFromAdmin/484f8ac3504f985fdafb904759fb56ee/family1/235af62437d94a4aac6c151a20a173b3 2023-03-05 19:56:25,588 INFO [Listener at localhost/40787] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e202303052140dfccc8d94383b42a1106b0b76387_del 2023-03-05 19:56:25,588 INFO [Listener at localhost/40787] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303052140dfccc8d94383b42a1106b0b76387_del to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e202303052140dfccc8d94383b42a1106b0b76387_del 2023-03-05 19:56:26,400 INFO [Listener at localhost/40787] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 48c405b23f984649b77bc5bfd528a783 2023-03-05 19:56:26,401 DEBUG [Listener at localhost/40787] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/484f8ac3504f985fdafb904759fb56ee/.tmp/family2/48c405b23f984649b77bc5bfd528a783 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/484f8ac3504f985fdafb904759fb56ee/family2/48c405b23f984649b77bc5bfd528a783 2023-03-05 19:56:26,404 INFO [Listener at localhost/40787] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 48c405b23f984649b77bc5bfd528a783 2023-03-05 19:56:26,405 INFO [Listener at localhost/40787] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 484f8ac3504f985fdafb904759fb56ee/family2 of 484f8ac3504f985fdafb904759fb56ee into 48c405b23f984649b77bc5bfd528a783(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-05 19:56:26,405 DEBUG [Listener at localhost/40787] regionserver.HRegion(2289): Compaction status journal for 484f8ac3504f985fdafb904759fb56ee: 2023-03-05 19:56:26,405 DEBUG [Listener at localhost/40787] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-05 19:56:26,405 DEBUG [Listener at localhost/40787] regionserver.HStore(1912): 27d678a97a95df40c894e3fe4540a1f2/family1 is initiating major compaction (all files) 2023-03-05 19:56:26,406 INFO [Listener at localhost/40787] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-05 19:56:26,406 INFO [Listener at localhost/40787] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-05 19:56:26,406 INFO [Listener at localhost/40787] regionserver.HRegion(2259): Starting compaction of 27d678a97a95df40c894e3fe4540a1f2/family1 in testMajorCompactionFromAdmin,2,1678046174038.27d678a97a95df40c894e3fe4540a1f2. 2023-03-05 19:56:26,406 INFO [Listener at localhost/40787] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/27d678a97a95df40c894e3fe4540a1f2/family1/b95506c2d50a46bf86830e6f67dda314, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/27d678a97a95df40c894e3fe4540a1f2/family1/3b82c3fc0bfa4990b0c9e011cd31d35a, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/27d678a97a95df40c894e3fe4540a1f2/family1/2ee46606b01b42ebafce97b03f725567, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/27d678a97a95df40c894e3fe4540a1f2/family1/e7913b8bbfcb454e88855230488cbb3f, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/27d678a97a95df40c894e3fe4540a1f2/family1/c86374b33b5145b08312cf08ad34c330] into tmpdir=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/27d678a97a95df40c894e3fe4540a1f2/.tmp, totalSize=27.1 K 2023-03-05 19:56:26,406 DEBUG [Listener at localhost/40787] compactions.Compactor(207): Compacting b95506c2d50a46bf86830e6f67dda314, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678046174700 2023-03-05 19:56:26,407 DEBUG [Listener at localhost/40787] compactions.Compactor(207): Compacting 3b82c3fc0bfa4990b0c9e011cd31d35a, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678046176628 2023-03-05 19:56:26,407 DEBUG [Listener at localhost/40787] compactions.Compactor(207): Compacting 2ee46606b01b42ebafce97b03f725567, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678046178512 2023-03-05 19:56:26,407 DEBUG [Listener at localhost/40787] compactions.Compactor(207): Compacting e7913b8bbfcb454e88855230488cbb3f, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678046180367 2023-03-05 19:56:26,408 DEBUG [Listener at localhost/40787] compactions.Compactor(207): Compacting c86374b33b5145b08312cf08ad34c330, keycount=3, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-05 19:56:26,416 INFO [Listener at localhost/40787] throttle.PressureAwareThroughputController(145): 27d678a97a95df40c894e3fe4540a1f2#family1#compaction#199 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-05 19:56:26,482 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/484f8ac3504f985fdafb904759fb56ee/family2/3031bd61f0e5467a907106f8f0333070, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/484f8ac3504f985fdafb904759fb56ee/family2/52a339f9dbe24d748095616145b6069c, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/484f8ac3504f985fdafb904759fb56ee/family2/3f2cab9d93f24668a5948bdf58aae419, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/484f8ac3504f985fdafb904759fb56ee/family2/2a2ffad3470e41ed9d75c5ec3c7d57e9, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/484f8ac3504f985fdafb904759fb56ee/family2/637e42afeeb84981af879d9b012e8dae] to archive 2023-03-05 19:56:26,483 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-05 19:56:26,484 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/484f8ac3504f985fdafb904759fb56ee/family2/3031bd61f0e5467a907106f8f0333070 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionFromAdmin/484f8ac3504f985fdafb904759fb56ee/family2/3031bd61f0e5467a907106f8f0333070 2023-03-05 19:56:26,485 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/484f8ac3504f985fdafb904759fb56ee/family2/52a339f9dbe24d748095616145b6069c to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionFromAdmin/484f8ac3504f985fdafb904759fb56ee/family2/52a339f9dbe24d748095616145b6069c 2023-03-05 19:56:26,486 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/484f8ac3504f985fdafb904759fb56ee/family2/3f2cab9d93f24668a5948bdf58aae419 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionFromAdmin/484f8ac3504f985fdafb904759fb56ee/family2/3f2cab9d93f24668a5948bdf58aae419 2023-03-05 19:56:26,487 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/484f8ac3504f985fdafb904759fb56ee/family2/2a2ffad3470e41ed9d75c5ec3c7d57e9 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionFromAdmin/484f8ac3504f985fdafb904759fb56ee/family2/2a2ffad3470e41ed9d75c5ec3c7d57e9 2023-03-05 19:56:26,488 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/484f8ac3504f985fdafb904759fb56ee/family2/637e42afeeb84981af879d9b012e8dae to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionFromAdmin/484f8ac3504f985fdafb904759fb56ee/family2/637e42afeeb84981af879d9b012e8dae 2023-03-05 19:56:26,853 INFO [Listener at localhost/40787] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230305676cc07e295544769f83ae5f86678ddc_del 2023-03-05 19:56:26,854 INFO [Listener at localhost/40787] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230305676cc07e295544769f83ae5f86678ddc_del to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230305676cc07e295544769f83ae5f86678ddc_del 2023-03-05 19:56:27,306 INFO [Listener at localhost/40787] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9edf6ec7aa8c444fb6394c4028bf661e 2023-03-05 19:56:27,307 DEBUG [Listener at localhost/40787] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/27d678a97a95df40c894e3fe4540a1f2/.tmp/family1/9edf6ec7aa8c444fb6394c4028bf661e as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/27d678a97a95df40c894e3fe4540a1f2/family1/9edf6ec7aa8c444fb6394c4028bf661e 2023-03-05 19:56:27,329 INFO [Listener at localhost/40787] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9edf6ec7aa8c444fb6394c4028bf661e 2023-03-05 19:56:27,330 INFO [Listener at localhost/40787] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 27d678a97a95df40c894e3fe4540a1f2/family1 of 27d678a97a95df40c894e3fe4540a1f2 into 9edf6ec7aa8c444fb6394c4028bf661e(size=7.0 K), total size for store is 7.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-05 19:56:27,330 DEBUG [Listener at localhost/40787] regionserver.HRegion(2289): Compaction status journal for 27d678a97a95df40c894e3fe4540a1f2: 2023-03-05 19:56:27,330 DEBUG [Listener at localhost/40787] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-05 19:56:27,330 DEBUG [Listener at localhost/40787] regionserver.HStore(1912): 27d678a97a95df40c894e3fe4540a1f2/family2 is initiating major compaction (all files) 2023-03-05 19:56:27,330 INFO [Listener at localhost/40787] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-05 19:56:27,330 INFO [Listener at localhost/40787] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-05 19:56:27,332 INFO [Listener at localhost/40787] regionserver.HRegion(2259): Starting compaction of 27d678a97a95df40c894e3fe4540a1f2/family2 in testMajorCompactionFromAdmin,2,1678046174038.27d678a97a95df40c894e3fe4540a1f2. 2023-03-05 19:56:27,332 INFO [Listener at localhost/40787] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/27d678a97a95df40c894e3fe4540a1f2/family2/52ea01697ce04731a85b6040cee8e0d2, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/27d678a97a95df40c894e3fe4540a1f2/family2/a51cee78384b4559ae358ceb3fc3a35a, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/27d678a97a95df40c894e3fe4540a1f2/family2/0fe6bf4b584a4bdcaaa0d012a328c8d8, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/27d678a97a95df40c894e3fe4540a1f2/family2/0f0187e0342542c296b9416a59434cde, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/27d678a97a95df40c894e3fe4540a1f2/family2/90f910f718734a4ab1f6602bbdf7d60e] into tmpdir=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/27d678a97a95df40c894e3fe4540a1f2/.tmp, totalSize=25.4 K 2023-03-05 19:56:27,333 DEBUG [Listener at localhost/40787] compactions.Compactor(207): Compacting 52ea01697ce04731a85b6040cee8e0d2, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678046174700 2023-03-05 19:56:27,333 DEBUG [Listener at localhost/40787] compactions.Compactor(207): Compacting a51cee78384b4559ae358ceb3fc3a35a, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678046176628 2023-03-05 19:56:27,333 DEBUG [Listener at localhost/40787] compactions.Compactor(207): Compacting 0fe6bf4b584a4bdcaaa0d012a328c8d8, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678046178512 2023-03-05 19:56:27,334 DEBUG [Listener at localhost/40787] compactions.Compactor(207): Compacting 0f0187e0342542c296b9416a59434cde, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678046180367 2023-03-05 19:56:27,334 DEBUG [Listener at localhost/40787] compactions.Compactor(207): Compacting 90f910f718734a4ab1f6602bbdf7d60e, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-05 19:56:27,341 INFO [Listener at localhost/40787] throttle.PressureAwareThroughputController(145): 27d678a97a95df40c894e3fe4540a1f2#family2#compaction#200 average throughput is 0.53 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-05 19:56:27,382 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/27d678a97a95df40c894e3fe4540a1f2/family1/b95506c2d50a46bf86830e6f67dda314, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/27d678a97a95df40c894e3fe4540a1f2/family1/3b82c3fc0bfa4990b0c9e011cd31d35a, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/27d678a97a95df40c894e3fe4540a1f2/family1/2ee46606b01b42ebafce97b03f725567, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/27d678a97a95df40c894e3fe4540a1f2/family1/e7913b8bbfcb454e88855230488cbb3f, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/27d678a97a95df40c894e3fe4540a1f2/family1/c86374b33b5145b08312cf08ad34c330] to archive 2023-03-05 19:56:27,383 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-05 19:56:27,384 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/27d678a97a95df40c894e3fe4540a1f2/family1/b95506c2d50a46bf86830e6f67dda314 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionFromAdmin/27d678a97a95df40c894e3fe4540a1f2/family1/b95506c2d50a46bf86830e6f67dda314 2023-03-05 19:56:27,385 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/27d678a97a95df40c894e3fe4540a1f2/family1/3b82c3fc0bfa4990b0c9e011cd31d35a to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionFromAdmin/27d678a97a95df40c894e3fe4540a1f2/family1/3b82c3fc0bfa4990b0c9e011cd31d35a 2023-03-05 19:56:27,386 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/27d678a97a95df40c894e3fe4540a1f2/family1/2ee46606b01b42ebafce97b03f725567 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionFromAdmin/27d678a97a95df40c894e3fe4540a1f2/family1/2ee46606b01b42ebafce97b03f725567 2023-03-05 19:56:27,387 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/27d678a97a95df40c894e3fe4540a1f2/family1/e7913b8bbfcb454e88855230488cbb3f to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionFromAdmin/27d678a97a95df40c894e3fe4540a1f2/family1/e7913b8bbfcb454e88855230488cbb3f 2023-03-05 19:56:27,388 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/27d678a97a95df40c894e3fe4540a1f2/family1/c86374b33b5145b08312cf08ad34c330 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionFromAdmin/27d678a97a95df40c894e3fe4540a1f2/family1/c86374b33b5145b08312cf08ad34c330 2023-03-05 19:56:27,748 INFO [Listener at localhost/40787] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230305c4e41716448440c19e5661595503ab32_del 2023-03-05 19:56:27,748 INFO [Listener at localhost/40787] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230305c4e41716448440c19e5661595503ab32_del to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c20230305c4e41716448440c19e5661595503ab32_del 2023-03-05 19:56:28,161 INFO [Listener at localhost/40787] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 826275df90c34afb922c271fd59e7568 2023-03-05 19:56:28,162 DEBUG [Listener at localhost/40787] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/27d678a97a95df40c894e3fe4540a1f2/.tmp/family2/826275df90c34afb922c271fd59e7568 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/27d678a97a95df40c894e3fe4540a1f2/family2/826275df90c34afb922c271fd59e7568 2023-03-05 19:56:28,165 INFO [Listener at localhost/40787] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 826275df90c34afb922c271fd59e7568 2023-03-05 19:56:28,166 INFO [Listener at localhost/40787] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 27d678a97a95df40c894e3fe4540a1f2/family2 of 27d678a97a95df40c894e3fe4540a1f2 into 826275df90c34afb922c271fd59e7568(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-05 19:56:28,166 DEBUG [Listener at localhost/40787] regionserver.HRegion(2289): Compaction status journal for 27d678a97a95df40c894e3fe4540a1f2: 2023-03-05 19:56:28,166 DEBUG [Listener at localhost/40787] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-05 19:56:28,166 DEBUG [Listener at localhost/40787] regionserver.HStore(1912): 806fed504069a9293ba794165a04fd64/family1 is initiating major compaction (all files) 2023-03-05 19:56:28,166 INFO [Listener at localhost/40787] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-05 19:56:28,166 INFO [Listener at localhost/40787] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-05 19:56:28,166 INFO [Listener at localhost/40787] regionserver.HRegion(2259): Starting compaction of 806fed504069a9293ba794165a04fd64/family1 in testMajorCompactionFromAdmin,1,1678046174038.806fed504069a9293ba794165a04fd64. 2023-03-05 19:56:28,166 INFO [Listener at localhost/40787] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/806fed504069a9293ba794165a04fd64/family1/76f1d4eac62f464dafe23cb47b0fa058, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/806fed504069a9293ba794165a04fd64/family1/f048033a76f842c2ab3d1f55862f4751, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/806fed504069a9293ba794165a04fd64/family1/297eb1800270488ea5afc665defca8d0, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/806fed504069a9293ba794165a04fd64/family1/d95d39f65ea04368a1051a20fc118ea8, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/806fed504069a9293ba794165a04fd64/family1/113c92cea7d64dd5b203b7c519cfa62c] into tmpdir=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/806fed504069a9293ba794165a04fd64/.tmp, totalSize=27.1 K 2023-03-05 19:56:28,167 DEBUG [Listener at localhost/40787] compactions.Compactor(207): Compacting 76f1d4eac62f464dafe23cb47b0fa058, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678046174700 2023-03-05 19:56:28,167 DEBUG [Listener at localhost/40787] compactions.Compactor(207): Compacting f048033a76f842c2ab3d1f55862f4751, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678046176627 2023-03-05 19:56:28,167 DEBUG [Listener at localhost/40787] compactions.Compactor(207): Compacting 297eb1800270488ea5afc665defca8d0, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678046178512 2023-03-05 19:56:28,168 DEBUG [Listener at localhost/40787] compactions.Compactor(207): Compacting d95d39f65ea04368a1051a20fc118ea8, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678046180367 2023-03-05 19:56:28,168 DEBUG [Listener at localhost/40787] compactions.Compactor(207): Compacting 113c92cea7d64dd5b203b7c519cfa62c, keycount=3, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-05 19:56:28,175 INFO [Listener at localhost/40787] throttle.PressureAwareThroughputController(145): 806fed504069a9293ba794165a04fd64#family1#compaction#201 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-05 19:56:28,180 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/27d678a97a95df40c894e3fe4540a1f2/family2/52ea01697ce04731a85b6040cee8e0d2, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/27d678a97a95df40c894e3fe4540a1f2/family2/a51cee78384b4559ae358ceb3fc3a35a, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/27d678a97a95df40c894e3fe4540a1f2/family2/0fe6bf4b584a4bdcaaa0d012a328c8d8, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/27d678a97a95df40c894e3fe4540a1f2/family2/0f0187e0342542c296b9416a59434cde, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/27d678a97a95df40c894e3fe4540a1f2/family2/90f910f718734a4ab1f6602bbdf7d60e] to archive 2023-03-05 19:56:28,181 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-05 19:56:28,182 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/27d678a97a95df40c894e3fe4540a1f2/family2/52ea01697ce04731a85b6040cee8e0d2 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionFromAdmin/27d678a97a95df40c894e3fe4540a1f2/family2/52ea01697ce04731a85b6040cee8e0d2 2023-03-05 19:56:28,183 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/27d678a97a95df40c894e3fe4540a1f2/family2/a51cee78384b4559ae358ceb3fc3a35a to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionFromAdmin/27d678a97a95df40c894e3fe4540a1f2/family2/a51cee78384b4559ae358ceb3fc3a35a 2023-03-05 19:56:28,184 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/27d678a97a95df40c894e3fe4540a1f2/family2/0fe6bf4b584a4bdcaaa0d012a328c8d8 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionFromAdmin/27d678a97a95df40c894e3fe4540a1f2/family2/0fe6bf4b584a4bdcaaa0d012a328c8d8 2023-03-05 19:56:28,185 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/27d678a97a95df40c894e3fe4540a1f2/family2/0f0187e0342542c296b9416a59434cde to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionFromAdmin/27d678a97a95df40c894e3fe4540a1f2/family2/0f0187e0342542c296b9416a59434cde 2023-03-05 19:56:28,185 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/27d678a97a95df40c894e3fe4540a1f2/family2/90f910f718734a4ab1f6602bbdf7d60e to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionFromAdmin/27d678a97a95df40c894e3fe4540a1f2/family2/90f910f718734a4ab1f6602bbdf7d60e 2023-03-05 19:56:28,613 INFO [Listener at localhost/40787] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b2023030524ef201027ae406f9e02600824fbe4e3_del 2023-03-05 19:56:28,614 INFO [Listener at localhost/40787] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023030524ef201027ae406f9e02600824fbe4e3_del to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b2023030524ef201027ae406f9e02600824fbe4e3_del 2023-03-05 19:56:29,476 INFO [Listener at localhost/40787] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ea6aae56fead4d0698562336218a336e 2023-03-05 19:56:29,477 DEBUG [Listener at localhost/40787] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/806fed504069a9293ba794165a04fd64/.tmp/family1/ea6aae56fead4d0698562336218a336e as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/806fed504069a9293ba794165a04fd64/family1/ea6aae56fead4d0698562336218a336e 2023-03-05 19:56:29,499 INFO [Listener at localhost/40787] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ea6aae56fead4d0698562336218a336e 2023-03-05 19:56:29,500 INFO [Listener at localhost/40787] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 806fed504069a9293ba794165a04fd64/family1 of 806fed504069a9293ba794165a04fd64 into ea6aae56fead4d0698562336218a336e(size=7.0 K), total size for store is 7.0 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-05 19:56:29,501 DEBUG [Listener at localhost/40787] regionserver.HRegion(2289): Compaction status journal for 806fed504069a9293ba794165a04fd64: 2023-03-05 19:56:29,501 DEBUG [Listener at localhost/40787] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-05 19:56:29,501 DEBUG [Listener at localhost/40787] regionserver.HStore(1912): 806fed504069a9293ba794165a04fd64/family2 is initiating major compaction (all files) 2023-03-05 19:56:29,501 INFO [Listener at localhost/40787] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-05 19:56:29,501 INFO [Listener at localhost/40787] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-05 19:56:29,501 INFO [Listener at localhost/40787] regionserver.HRegion(2259): Starting compaction of 806fed504069a9293ba794165a04fd64/family2 in testMajorCompactionFromAdmin,1,1678046174038.806fed504069a9293ba794165a04fd64. 2023-03-05 19:56:29,501 INFO [Listener at localhost/40787] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/806fed504069a9293ba794165a04fd64/family2/d850c9c1ee784e71a5fbb3fc2927b071, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/806fed504069a9293ba794165a04fd64/family2/fb21e0ddcdfe49a188a2f30bd3074b5e, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/806fed504069a9293ba794165a04fd64/family2/8f022f0990784143bd64e4161d6ca99d, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/806fed504069a9293ba794165a04fd64/family2/5eff69a5399a48188ba32d73e7d90955, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/806fed504069a9293ba794165a04fd64/family2/694c4424907e47d294f0564a7ba4db41] into tmpdir=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/806fed504069a9293ba794165a04fd64/.tmp, totalSize=25.4 K 2023-03-05 19:56:29,501 DEBUG [Listener at localhost/40787] compactions.Compactor(207): Compacting d850c9c1ee784e71a5fbb3fc2927b071, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678046174700 2023-03-05 19:56:29,502 DEBUG [Listener at localhost/40787] compactions.Compactor(207): Compacting fb21e0ddcdfe49a188a2f30bd3074b5e, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678046176627 2023-03-05 19:56:29,502 DEBUG [Listener at localhost/40787] compactions.Compactor(207): Compacting 8f022f0990784143bd64e4161d6ca99d, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678046178512 2023-03-05 19:56:29,502 DEBUG [Listener at localhost/40787] compactions.Compactor(207): Compacting 5eff69a5399a48188ba32d73e7d90955, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678046180367 2023-03-05 19:56:29,502 DEBUG [Listener at localhost/40787] compactions.Compactor(207): Compacting 694c4424907e47d294f0564a7ba4db41, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-05 19:56:29,510 INFO [Listener at localhost/40787] throttle.PressureAwareThroughputController(145): 806fed504069a9293ba794165a04fd64#family2#compaction#202 average throughput is 0.53 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-05 19:56:29,580 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/806fed504069a9293ba794165a04fd64/family1/76f1d4eac62f464dafe23cb47b0fa058, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/806fed504069a9293ba794165a04fd64/family1/f048033a76f842c2ab3d1f55862f4751, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/806fed504069a9293ba794165a04fd64/family1/297eb1800270488ea5afc665defca8d0, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/806fed504069a9293ba794165a04fd64/family1/d95d39f65ea04368a1051a20fc118ea8, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/806fed504069a9293ba794165a04fd64/family1/113c92cea7d64dd5b203b7c519cfa62c] to archive 2023-03-05 19:56:29,581 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-05 19:56:29,583 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/806fed504069a9293ba794165a04fd64/family1/76f1d4eac62f464dafe23cb47b0fa058 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionFromAdmin/806fed504069a9293ba794165a04fd64/family1/76f1d4eac62f464dafe23cb47b0fa058 2023-03-05 19:56:29,584 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/806fed504069a9293ba794165a04fd64/family1/f048033a76f842c2ab3d1f55862f4751 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionFromAdmin/806fed504069a9293ba794165a04fd64/family1/f048033a76f842c2ab3d1f55862f4751 2023-03-05 19:56:29,585 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/806fed504069a9293ba794165a04fd64/family1/297eb1800270488ea5afc665defca8d0 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionFromAdmin/806fed504069a9293ba794165a04fd64/family1/297eb1800270488ea5afc665defca8d0 2023-03-05 19:56:29,586 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/806fed504069a9293ba794165a04fd64/family1/d95d39f65ea04368a1051a20fc118ea8 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionFromAdmin/806fed504069a9293ba794165a04fd64/family1/d95d39f65ea04368a1051a20fc118ea8 2023-03-05 19:56:29,587 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/806fed504069a9293ba794165a04fd64/family1/113c92cea7d64dd5b203b7c519cfa62c to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionFromAdmin/806fed504069a9293ba794165a04fd64/family1/113c92cea7d64dd5b203b7c519cfa62c 2023-03-05 19:56:29,917 INFO [Listener at localhost/40787] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b202303057f2806676523495494547575a5153400_del 2023-03-05 19:56:29,917 INFO [Listener at localhost/40787] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303057f2806676523495494547575a5153400_del to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b202303057f2806676523495494547575a5153400_del 2023-03-05 19:56:30,730 INFO [Listener at localhost/40787] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 24d2ce111eae4214a6a3722f95c9757c 2023-03-05 19:56:30,731 DEBUG [Listener at localhost/40787] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/806fed504069a9293ba794165a04fd64/.tmp/family2/24d2ce111eae4214a6a3722f95c9757c as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/806fed504069a9293ba794165a04fd64/family2/24d2ce111eae4214a6a3722f95c9757c 2023-03-05 19:56:30,734 INFO [Listener at localhost/40787] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 24d2ce111eae4214a6a3722f95c9757c 2023-03-05 19:56:30,734 INFO [Listener at localhost/40787] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 806fed504069a9293ba794165a04fd64/family2 of 806fed504069a9293ba794165a04fd64 into 24d2ce111eae4214a6a3722f95c9757c(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-05 19:56:30,735 DEBUG [Listener at localhost/40787] regionserver.HRegion(2289): Compaction status journal for 806fed504069a9293ba794165a04fd64: 2023-03-05 19:56:30,738 DEBUG [Listener at localhost/40787] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-05 19:56:30,741 DEBUG [Listener at localhost/40787] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-05 19:56:30,745 DEBUG [Listener at localhost/40787] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-05 19:56:30,745 DEBUG [Listener at localhost/40787] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-05 19:56:30,750 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=38221] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10\xCB\xA0\xE4\xC9z\xCF\xFF+\x1FU\xE1|\xD4\xB2#\x14"\x10F0\xDA\xDEX\xBEDK,1\xF2\xE1\x11\x8C\x8F\xEE*\x10V\xB4q\x5CP\x03\x05\x1F/\xCA\x00\x5C{c\x8Bh2\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}] of the table testMajorCompactionFromAdmin 2023-03-05 19:56:30,780 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/806fed504069a9293ba794165a04fd64/family2/d850c9c1ee784e71a5fbb3fc2927b071, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/806fed504069a9293ba794165a04fd64/family2/fb21e0ddcdfe49a188a2f30bd3074b5e, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/806fed504069a9293ba794165a04fd64/family2/8f022f0990784143bd64e4161d6ca99d, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/806fed504069a9293ba794165a04fd64/family2/5eff69a5399a48188ba32d73e7d90955, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/806fed504069a9293ba794165a04fd64/family2/694c4424907e47d294f0564a7ba4db41] to archive 2023-03-05 19:56:30,781 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-05 19:56:30,781 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] procedure2.ProcedureExecutor(1029): Stored pid=123, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE 2023-03-05 19:56:30,782 DEBUG [PEWorker-4] locking.LockProcedure(309): LOCKED pid=123, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE 2023-03-05 19:56:30,783 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionFromAdmin', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-05 19:56:30,783 INFO [PEWorker-4] procedure2.TimeoutExecutorThread(81): ADDED pid=123, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678046790783 2023-03-05 19:56:30,783 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/806fed504069a9293ba794165a04fd64/family2/d850c9c1ee784e71a5fbb3fc2927b071 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionFromAdmin/806fed504069a9293ba794165a04fd64/family2/d850c9c1ee784e71a5fbb3fc2927b071 2023-03-05 19:56:30,783 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-05 19:56:30,784 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/806fed504069a9293ba794165a04fd64/family2/fb21e0ddcdfe49a188a2f30bd3074b5e to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionFromAdmin/806fed504069a9293ba794165a04fd64/family2/fb21e0ddcdfe49a188a2f30bd3074b5e 2023-03-05 19:56:30,785 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/806fed504069a9293ba794165a04fd64/family2/8f022f0990784143bd64e4161d6ca99d to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionFromAdmin/806fed504069a9293ba794165a04fd64/family2/8f022f0990784143bd64e4161d6ca99d 2023-03-05 19:56:30,786 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/806fed504069a9293ba794165a04fd64/family2/5eff69a5399a48188ba32d73e7d90955 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionFromAdmin/806fed504069a9293ba794165a04fd64/family2/5eff69a5399a48188ba32d73e7d90955 2023-03-05 19:56:30,787 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/806fed504069a9293ba794165a04fd64/family2/694c4424907e47d294f0564a7ba4db41 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionFromAdmin/806fed504069a9293ba794165a04fd64/family2/694c4424907e47d294f0564a7ba4db41 2023-03-05 19:56:30,840 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-05 19:56:30,840 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x7048fa16 to 127.0.0.1:62957 2023-03-05 19:56:30,840 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-05 19:56:31,137 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 3 del files, 12 selected files, and 0 irrelevant files table 'testMajorCompactionFromAdmin' and column 'family1' 2023-03-05 19:56:31,159 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20230305d507491929fc4e0da0c8abe9c0c57e6f_del 2023-03-05 19:56:31,181 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b2023030524ef201027ae406f9e02600824fbe4e3_del 2023-03-05 19:56:31,202 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230305676cc07e295544769f83ae5f86678ddc_del 2023-03-05 19:56:31,202 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] compactions.PartitionedMobCompactor(350): After merging, there are 3 del files. table='testMajorCompactionFromAdmin' column='family1' 2023-03-05 19:56:31,203 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] zookeeper.ReadOnlyZKClient(139): Connect 0x0646a14f to 127.0.0.1:62957 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-05 19:56:31,207 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@47086e4f, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-05 19:56:31,208 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230305 for table='testMajorCompactionFromAdmin' and column='family1' 2023-03-05 19:56:31,336 DEBUG [regionserver/jenkins-hbase4:0.Chore.2] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-03-05 19:56:31,801 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023030539219d3b7c854e41a9cd1a1e9fcbeb21 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e2023030539219d3b7c854e41a9cd1a1e9fcbeb21 2023-03-05 19:56:31,802 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230305 table='testMajorCompactionFromAdmin' column='family1' 2023-03-05 19:56:31,813 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-05 19:56:31,816 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:32804, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-05 19:56:31,824 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-05 19:56:31,826 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:34816, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-05 19:56:31,855 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230305/family1/58310bdc81be4d55a5fdd63f9a64d0f7 first=Optional[0\x00\x00\x00\x01] last=Optional[0\x00\x00\x00\x07] 2023-03-05 19:56:31,856 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionFromAdmin,,1678046174038.484f8ac3504f985fdafb904759fb56ee., hostname=jenkins-hbase4.apache.org,37037,1678046018983, seqNum=2 for row with hfile group [{family1,hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230305/family1/58310bdc81be4d55a5fdd63f9a64d0f7}] 2023-03-05 19:56:31,860 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.HStore(763): Validating hfile at hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230305/family1/58310bdc81be4d55a5fdd63f9a64d0f7 for inclusion in 484f8ac3504f985fdafb904759fb56ee/family1 2023-03-05 19:56:31,881 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.HStore(775): HFile bounds: first=0\x00\x00\x00\x01 last=0\x00\x00\x00\x07 2023-03-05 19:56:31,882 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.HStore(777): Region bounds: first= last=1 2023-03-05 19:56:31,882 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.HRegion(2446): Flush status journal for 484f8ac3504f985fdafb904759fb56ee: 2023-03-05 19:56:31,883 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230305/family1/58310bdc81be4d55a5fdd63f9a64d0f7 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testMajorCompactionFromAdmin__v5tm448al4k4ircrvqtd5to1fapcvdl30a38lnavds067s9kmlbhei8amubt9cmi/family1/58310bdc81be4d55a5fdd63f9a64d0f7 2023-03-05 19:56:31,885 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testMajorCompactionFromAdmin__v5tm448al4k4ircrvqtd5to1fapcvdl30a38lnavds067s9kmlbhei8amubt9cmi/family1/58310bdc81be4d55a5fdd63f9a64d0f7 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/484f8ac3504f985fdafb904759fb56ee/family1/8ed86c401cbf4a4ea8451435e39f73f8_SeqId_28_ 2023-03-05 19:56:31,885 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.HStore(849): Loaded HFile hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testMajorCompactionFromAdmin__v5tm448al4k4ircrvqtd5to1fapcvdl30a38lnavds067s9kmlbhei8amubt9cmi/family1/58310bdc81be4d55a5fdd63f9a64d0f7 into 484f8ac3504f985fdafb904759fb56ee/family1 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/484f8ac3504f985fdafb904759fb56ee/family1/8ed86c401cbf4a4ea8451435e39f73f8_SeqId_28_ - updating store file list. 2023-03-05 19:56:31,909 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.HStore(882): Loaded HFile hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/484f8ac3504f985fdafb904759fb56ee/family1/8ed86c401cbf4a4ea8451435e39f73f8_SeqId_28_ into 484f8ac3504f985fdafb904759fb56ee/family1 2023-03-05 19:56:31,909 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.HStore(855): Successfully loaded hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testMajorCompactionFromAdmin__v5tm448al4k4ircrvqtd5to1fapcvdl30a38lnavds067s9kmlbhei8amubt9cmi/family1/58310bdc81be4d55a5fdd63f9a64d0f7 into 484f8ac3504f985fdafb904759fb56ee/family1 (new location: hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/484f8ac3504f985fdafb904759fb56ee/family1/8ed86c401cbf4a4ea8451435e39f73f8_SeqId_28_) 2023-03-05 19:56:31,909 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testMajorCompactionFromAdmin__v5tm448al4k4ircrvqtd5to1fapcvdl30a38lnavds067s9kmlbhei8amubt9cmi/family1/58310bdc81be4d55a5fdd63f9a64d0f7 2023-03-05 19:56:31,911 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230305 table='testMajorCompactionFromAdmin' column='family1' 2023-03-05 19:56:31,912 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-05 19:56:31,913 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e2023030524b5b5e287994efc8204f27810fa8262 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e2023030524b5b5e287994efc8204f27810fa8262 2023-03-05 19:56:31,914 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e2023030547b5e4a18bdb437b99f5f85dbe33db3f to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e2023030547b5e4a18bdb437b99f5f85dbe33db3f 2023-03-05 19:56:31,915 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202303056f8ec686125e46d6840dd7a563cc76ea to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202303056f8ec686125e46d6840dd7a563cc76ea 2023-03-05 19:56:31,916 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230305dc111309c35549b582ac70838b1fc273 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230305dc111309c35549b582ac70838b1fc273 2023-03-05 19:56:31,916 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionFromAdmin 4 expired mob files are deleted 2023-03-05 19:56:31,916 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230305' 2023-03-05 19:56:31,916 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230305 for table='testMajorCompactionFromAdmin' and column='family1' 2023-03-05 19:56:31,916 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230305 for table='testMajorCompactionFromAdmin' and column='family1' 2023-03-05 19:56:32,463 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230305b2b7669b6454406c83a8b10842753629 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230305b2b7669b6454406c83a8b10842753629 2023-03-05 19:56:32,464 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230305 table='testMajorCompactionFromAdmin' column='family1' 2023-03-05 19:56:32,508 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230305/family1/9df2f11c1c2c4a9591910489e07936ea first=Optional[2\x00\x00\x00\x01] last=Optional[2\x00\x00\x00\x07] 2023-03-05 19:56:32,509 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionFromAdmin,2,1678046174038.27d678a97a95df40c894e3fe4540a1f2., hostname=jenkins-hbase4.apache.org,37037,1678046018983, seqNum=2 for row 2 with hfile group [{family1,hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230305/family1/9df2f11c1c2c4a9591910489e07936ea}] 2023-03-05 19:56:32,512 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.HStore(763): Validating hfile at hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230305/family1/9df2f11c1c2c4a9591910489e07936ea for inclusion in 27d678a97a95df40c894e3fe4540a1f2/family1 2023-03-05 19:56:32,533 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.HStore(775): HFile bounds: first=2\x00\x00\x00\x01 last=2\x00\x00\x00\x07 2023-03-05 19:56:32,533 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.HStore(777): Region bounds: first=2 last= 2023-03-05 19:56:32,533 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.HRegion(2446): Flush status journal for 27d678a97a95df40c894e3fe4540a1f2: 2023-03-05 19:56:32,533 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230305/family1/9df2f11c1c2c4a9591910489e07936ea to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testMajorCompactionFromAdmin__erkammok8llb1l0rghj204esd58q5h9dvd1jjfok5sonsda479baqpgi7gltnr/family1/9df2f11c1c2c4a9591910489e07936ea 2023-03-05 19:56:32,535 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testMajorCompactionFromAdmin__erkammok8llb1l0rghj204esd58q5h9dvd1jjfok5sonsda479baqpgi7gltnr/family1/9df2f11c1c2c4a9591910489e07936ea as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/27d678a97a95df40c894e3fe4540a1f2/family1/d5740dcccc3c490d9cc4e60936d84002_SeqId_28_ 2023-03-05 19:56:32,536 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.HStore(849): Loaded HFile hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testMajorCompactionFromAdmin__erkammok8llb1l0rghj204esd58q5h9dvd1jjfok5sonsda479baqpgi7gltnr/family1/9df2f11c1c2c4a9591910489e07936ea into 27d678a97a95df40c894e3fe4540a1f2/family1 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/27d678a97a95df40c894e3fe4540a1f2/family1/d5740dcccc3c490d9cc4e60936d84002_SeqId_28_ - updating store file list. 2023-03-05 19:56:32,557 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.HStore(882): Loaded HFile hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/27d678a97a95df40c894e3fe4540a1f2/family1/d5740dcccc3c490d9cc4e60936d84002_SeqId_28_ into 27d678a97a95df40c894e3fe4540a1f2/family1 2023-03-05 19:56:32,557 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.HStore(855): Successfully loaded hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testMajorCompactionFromAdmin__erkammok8llb1l0rghj204esd58q5h9dvd1jjfok5sonsda479baqpgi7gltnr/family1/9df2f11c1c2c4a9591910489e07936ea into 27d678a97a95df40c894e3fe4540a1f2/family1 (new location: hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/27d678a97a95df40c894e3fe4540a1f2/family1/d5740dcccc3c490d9cc4e60936d84002_SeqId_28_) 2023-03-05 19:56:32,558 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testMajorCompactionFromAdmin__erkammok8llb1l0rghj204esd58q5h9dvd1jjfok5sonsda479baqpgi7gltnr/family1/9df2f11c1c2c4a9591910489e07936ea 2023-03-05 19:56:32,559 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230305 table='testMajorCompactionFromAdmin' column='family1' 2023-03-05 19:56:32,560 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-05 19:56:32,561 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202303053ec254fceddf4af78c1846209703f42e to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202303053ec254fceddf4af78c1846209703f42e 2023-03-05 19:56:32,562 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202303054dc539352e8843ce948b0ea4502741a4 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202303054dc539352e8843ce948b0ea4502741a4 2023-03-05 19:56:32,563 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230305b11ae57e62354bb09d12e0a7a06e4000 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230305b11ae57e62354bb09d12e0a7a06e4000 2023-03-05 19:56:32,564 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230305eef0ba6b6aa5445c8043212cee716589 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230305eef0ba6b6aa5445c8043212cee716589 2023-03-05 19:56:32,564 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionFromAdmin 4 expired mob files are deleted 2023-03-05 19:56:32,564 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230305' 2023-03-05 19:56:32,564 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230305 for table='testMajorCompactionFromAdmin' and column='family1' 2023-03-05 19:56:32,564 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c4ca4238a0b923820dcc509a6f75849b20230305 for table='testMajorCompactionFromAdmin' and column='family1' 2023-03-05 19:56:32,579 DEBUG [regionserver/jenkins-hbase4:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-03-05 19:56:33,111 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303055f1e77e619eb43808b6942545bc850f3 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202303055f1e77e619eb43808b6942545bc850f3 2023-03-05 19:56:33,112 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230305 table='testMajorCompactionFromAdmin' column='family1' 2023-03-05 19:56:33,153 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230305/family1/f07f25b8aa104da7aeaf324c1a6200b1 first=Optional[1\x00\x00\x00\x01] last=Optional[1\x00\x00\x00\x07] 2023-03-05 19:56:33,154 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionFromAdmin,1,1678046174038.806fed504069a9293ba794165a04fd64., hostname=jenkins-hbase4.apache.org,37037,1678046018983, seqNum=2 for row 1 with hfile group [{family1,hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230305/family1/f07f25b8aa104da7aeaf324c1a6200b1}] 2023-03-05 19:56:33,156 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.HStore(763): Validating hfile at hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230305/family1/f07f25b8aa104da7aeaf324c1a6200b1 for inclusion in 806fed504069a9293ba794165a04fd64/family1 2023-03-05 19:56:33,177 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.HStore(775): HFile bounds: first=1\x00\x00\x00\x01 last=1\x00\x00\x00\x07 2023-03-05 19:56:33,177 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.HStore(777): Region bounds: first=1 last=2 2023-03-05 19:56:33,177 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.HRegion(2446): Flush status journal for 806fed504069a9293ba794165a04fd64: 2023-03-05 19:56:33,178 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230305/family1/f07f25b8aa104da7aeaf324c1a6200b1 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testMajorCompactionFromAdmin__k4574ic9s9jhak80fpju12rutobgncu3cujnhglri9kae4r09kld5dmn427f26rj/family1/f07f25b8aa104da7aeaf324c1a6200b1 2023-03-05 19:56:33,180 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testMajorCompactionFromAdmin__k4574ic9s9jhak80fpju12rutobgncu3cujnhglri9kae4r09kld5dmn427f26rj/family1/f07f25b8aa104da7aeaf324c1a6200b1 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/806fed504069a9293ba794165a04fd64/family1/b37ac8f1c18b42b4b386ed50a4b1178c_SeqId_28_ 2023-03-05 19:56:33,180 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.HStore(849): Loaded HFile hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testMajorCompactionFromAdmin__k4574ic9s9jhak80fpju12rutobgncu3cujnhglri9kae4r09kld5dmn427f26rj/family1/f07f25b8aa104da7aeaf324c1a6200b1 into 806fed504069a9293ba794165a04fd64/family1 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/806fed504069a9293ba794165a04fd64/family1/b37ac8f1c18b42b4b386ed50a4b1178c_SeqId_28_ - updating store file list. 2023-03-05 19:56:33,213 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.HStore(882): Loaded HFile hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/806fed504069a9293ba794165a04fd64/family1/b37ac8f1c18b42b4b386ed50a4b1178c_SeqId_28_ into 806fed504069a9293ba794165a04fd64/family1 2023-03-05 19:56:33,213 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.HStore(855): Successfully loaded hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testMajorCompactionFromAdmin__k4574ic9s9jhak80fpju12rutobgncu3cujnhglri9kae4r09kld5dmn427f26rj/family1/f07f25b8aa104da7aeaf324c1a6200b1 into 806fed504069a9293ba794165a04fd64/family1 (new location: hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/806fed504069a9293ba794165a04fd64/family1/b37ac8f1c18b42b4b386ed50a4b1178c_SeqId_28_) 2023-03-05 19:56:33,214 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37037] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/staging/jenkins__testMajorCompactionFromAdmin__k4574ic9s9jhak80fpju12rutobgncu3cujnhglri9kae4r09kld5dmn427f26rj/family1/f07f25b8aa104da7aeaf324c1a6200b1 2023-03-05 19:56:33,215 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230305 table='testMajorCompactionFromAdmin' column='family1' 2023-03-05 19:56:33,216 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-05 19:56:33,217 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b2023030553bf5dc2a7a64ce59d0701205796ffeb to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b2023030553bf5dc2a7a64ce59d0701205796ffeb 2023-03-05 19:56:33,218 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230305588a97d843c94c12ae18c76a49eb3040 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230305588a97d843c94c12ae18c76a49eb3040 2023-03-05 19:56:33,219 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230305aa7bac24fda0413e83c0cfb39758a1ac to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230305aa7bac24fda0413e83c0cfb39758a1ac 2023-03-05 19:56:33,220 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230305e19b92a441f24b5c9b7841418f7111b5 to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230305e19b92a441f24b5c9b7841418f7111b5 2023-03-05 19:56:33,220 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionFromAdmin 4 expired mob files are deleted 2023-03-05 19:56:33,220 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230305' 2023-03-05 19:56:33,220 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c4ca4238a0b923820dcc509a6f75849b20230305 for table='testMajorCompactionFromAdmin' and column='family1' 2023-03-05 19:56:33,220 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] compactions.PartitionedMobCompactor(354): After compaction, there are 3 mob files. table='testMajorCompactionFromAdmin' column='family1' 2023-03-05 19:56:33,220 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] compactions.PartitionedMobCompactor(364): After a mob compaction with all files selected, archiving the del files for table='testMajorCompactionFromAdmin' and column='family1' 2023-03-05 19:56:33,220 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] compactions.PartitionedMobCompactor(367): [hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230305d507491929fc4e0da0c8abe9c0c57e6f_del] 2023-03-05 19:56:33,221 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] backup.HFileArchiver(360): Archiving compacted files. 2023-03-05 19:56:33,222 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230305d507491929fc4e0da0c8abe9c0c57e6f_del to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230305d507491929fc4e0da0c8abe9c0c57e6f_del 2023-03-05 19:56:33,222 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] mob.MobUtils(508): Table testMajorCompactionFromAdmin 1 expired mob files are deleted 2023-03-05 19:56:33,222 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] compactions.PartitionedMobCompactor(367): [hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b2023030524ef201027ae406f9e02600824fbe4e3_del] 2023-03-05 19:56:33,222 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] backup.HFileArchiver(360): Archiving compacted files. 2023-03-05 19:56:33,223 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b2023030524ef201027ae406f9e02600824fbe4e3_del to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b2023030524ef201027ae406f9e02600824fbe4e3_del 2023-03-05 19:56:33,223 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] mob.MobUtils(508): Table testMajorCompactionFromAdmin 1 expired mob files are deleted 2023-03-05 19:56:33,223 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] compactions.PartitionedMobCompactor(367): [hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230305676cc07e295544769f83ae5f86678ddc_del] 2023-03-05 19:56:33,224 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] backup.HFileArchiver(360): Archiving compacted files. 2023-03-05 19:56:33,225 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230305676cc07e295544769f83ae5f86678ddc_del to hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230305676cc07e295544769f83ae5f86678ddc_del 2023-03-05 19:56:33,225 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] mob.MobUtils(508): Table testMajorCompactionFromAdmin 1 expired mob files are deleted 2023-03-05 19:56:33,225 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678046023222] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionFromAdmin', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-05 19:56:33,226 DEBUG [PEWorker-1] locking.LockProcedure(242): UNLOCKED pid=123, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE 2023-03-05 19:56:33,227 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=123, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE in 2.4460 sec 2023-03-05 19:56:33,240 DEBUG [Listener at localhost/40787] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-05 19:56:33,242 DEBUG [Listener at localhost/40787] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-05 19:56:33,244 DEBUG [Listener at localhost/40787] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-05 19:56:33,244 DEBUG [Listener at localhost/40787] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-05 19:56:33,248 INFO [Listener at localhost/40787] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-03-05 19:56:33,287 INFO [Listener at localhost/40787] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionFromAdmin Thread=248 (was 255), OpenFileDescriptor=629 (was 640), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=94 (was 83) - SystemLoadAverage LEAK? -, ProcessCount=171 (was 171), AvailableMemoryMB=4780 (was 5265) 2023-03-05 19:56:33,287 DEBUG [Listener at localhost/40787] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x7defec53 to 127.0.0.1:62957 2023-03-05 19:56:33,287 DEBUG [Listener at localhost/40787] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-05 19:56:33,288 INFO [Listener at localhost/40787] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-03-05 19:56:33,288 INFO [Listener at localhost/40787] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-05 19:56:33,288 DEBUG [Listener at localhost/40787] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x0629018b to 127.0.0.1:62957 2023-03-05 19:56:33,288 DEBUG [Listener at localhost/40787] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-05 19:56:33,288 DEBUG [Listener at localhost/40787] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-03-05 19:56:33,288 DEBUG [Listener at localhost/40787] util.JVMClusterUtil(257): Found active master hash=848934891, stopped=false 2023-03-05 19:56:33,288 INFO [Listener at localhost/40787] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,38221,1678046017738 2023-03-05 19:56:33,290 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-03-05 19:56:33,290 INFO [Listener at localhost/40787] procedure2.ProcedureExecutor(629): Stopping 2023-03-05 19:56:33,290 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-03-05 19:56:33,290 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-05 19:56:33,290 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-03-05 19:56:33,290 DEBUG [Listener at localhost/40787] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x04dc6a2d to 127.0.0.1:62957 2023-03-05 19:56:33,291 DEBUG [Listener at localhost/40787] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-05 19:56:33,291 INFO [Listener at localhost/40787] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,37037,1678046018983' ***** 2023-03-05 19:56:33,291 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-03-05 19:56:33,291 INFO [Listener at localhost/40787] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-03-05 19:56:33,291 INFO [RS:0;jenkins-hbase4:37037] regionserver.HeapMemoryManager(220): Stopping 2023-03-05 19:56:33,291 INFO [RS:0;jenkins-hbase4:37037] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-03-05 19:56:33,291 INFO [RS:0;jenkins-hbase4:37037] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-03-05 19:56:33,292 INFO [RS:0;jenkins-hbase4:37037] regionserver.HRegionServer(3304): Received CLOSE for 942afd2d0a4ceef025fd5a607956d53d 2023-03-05 19:56:33,292 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-03-05 19:56:33,293 INFO [RS:0;jenkins-hbase4:37037] regionserver.HRegionServer(3304): Received CLOSE for bf39f342faf613faca7f43b94cc73bc4 2023-03-05 19:56:33,293 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 942afd2d0a4ceef025fd5a607956d53d, disabling compactions & flushes 2023-03-05 19:56:33,293 INFO [RS:0;jenkins-hbase4:37037] regionserver.HRegionServer(3304): Received CLOSE for 25ed4806e34b655e8718967079efb560 2023-03-05 19:56:33,293 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,1,1678046116082.942afd2d0a4ceef025fd5a607956d53d. 2023-03-05 19:56:33,293 INFO [RS:0;jenkins-hbase4:37037] regionserver.HRegionServer(3304): Received CLOSE for f5738c13f081c7a8286ce16de3ec8a21 2023-03-05 19:56:33,293 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,1,1678046116082.942afd2d0a4ceef025fd5a607956d53d. 2023-03-05 19:56:33,293 INFO [RS:0;jenkins-hbase4:37037] regionserver.HRegionServer(3304): Received CLOSE for 6fdfc14e8c700cdbd0a252655cbff4b4 2023-03-05 19:56:33,293 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,1,1678046116082.942afd2d0a4ceef025fd5a607956d53d. after waiting 0 ms 2023-03-05 19:56:33,293 INFO [RS:0;jenkins-hbase4:37037] regionserver.HRegionServer(3304): Received CLOSE for 484f8ac3504f985fdafb904759fb56ee 2023-03-05 19:56:33,293 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,1,1678046116082.942afd2d0a4ceef025fd5a607956d53d. 2023-03-05 19:56:33,293 INFO [RS:0;jenkins-hbase4:37037] regionserver.HRegionServer(3304): Received CLOSE for 04217a89b4b7147d2fd1e001ac928c17 2023-03-05 19:56:33,293 INFO [RS:0;jenkins-hbase4:37037] regionserver.HRegionServer(3304): Received CLOSE for 319680d28f8da52640b97eff4eaa9239 2023-03-05 19:56:33,293 INFO [RS:0;jenkins-hbase4:37037] regionserver.HRegionServer(3304): Received CLOSE for aa3a9f808cf1db1e0667a9fff093c3ec 2023-03-05 19:56:33,293 INFO [RS:0;jenkins-hbase4:37037] regionserver.HRegionServer(3304): Received CLOSE for 25af79a83c9b248c956b57fc8507ead6 2023-03-05 19:56:33,293 INFO [RS:0;jenkins-hbase4:37037] regionserver.HRegionServer(3304): Received CLOSE for 014dc240b491a64066ab7b01d2b21258 2023-03-05 19:56:33,293 INFO [RS:0;jenkins-hbase4:37037] regionserver.HRegionServer(3304): Received CLOSE for 27d678a97a95df40c894e3fe4540a1f2 2023-03-05 19:56:33,293 INFO [RS:0;jenkins-hbase4:37037] regionserver.HRegionServer(3304): Received CLOSE for e6a63acce9d53b978339ac99badba6fa 2023-03-05 19:56:33,293 INFO [RS:0;jenkins-hbase4:37037] regionserver.HRegionServer(3304): Received CLOSE for 5f33741eb2f558bc4b10a65a4789a550 2023-03-05 19:56:33,293 INFO [RS:0;jenkins-hbase4:37037] regionserver.HRegionServer(3304): Received CLOSE for 237c9aa4b41fcfa45032528ec18afe9b 2023-03-05 19:56:33,293 INFO [RS:0;jenkins-hbase4:37037] regionserver.HRegionServer(3304): Received CLOSE for b9c878f961e0a32027f8960cfe26b4eb 2023-03-05 19:56:33,293 INFO [RS:0;jenkins-hbase4:37037] regionserver.HRegionServer(3304): Received CLOSE for 25c8546691095e1f6ab386a199ccadff 2023-03-05 19:56:33,293 INFO [RS:0;jenkins-hbase4:37037] regionserver.HRegionServer(3304): Received CLOSE for 2d86bbeed3970cf2266d199ff776cf8d 2023-03-05 19:56:33,293 INFO [RS:0;jenkins-hbase4:37037] regionserver.HRegionServer(3304): Received CLOSE for 806fed504069a9293ba794165a04fd64 2023-03-05 19:56:33,293 INFO [RS:0;jenkins-hbase4:37037] regionserver.HRegionServer(3304): Received CLOSE for daa57ebba7c140c319a2f6facc725e4a 2023-03-05 19:56:33,293 INFO [RS:0;jenkins-hbase4:37037] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:33,293 DEBUG [RS:0;jenkins-hbase4:37037] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x5aeef514 to 127.0.0.1:62957 2023-03-05 19:56:33,294 DEBUG [RS:0;jenkins-hbase4:37037] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-05 19:56:33,294 INFO [RS:0;jenkins-hbase4:37037] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-03-05 19:56:33,294 INFO [RS:0;jenkins-hbase4:37037] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-03-05 19:56:33,294 INFO [RS:0;jenkins-hbase4:37037] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-03-05 19:56:33,294 INFO [RS:0;jenkins-hbase4:37037] regionserver.HRegionServer(3304): Received CLOSE for 1588230740 2023-03-05 19:56:33,294 INFO [RS:0;jenkins-hbase4:37037] regionserver.HRegionServer(1475): Waiting on 21 regions to close 2023-03-05 19:56:33,294 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-03-05 19:56:33,295 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-03-05 19:56:33,295 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-03-05 19:56:33,295 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-03-05 19:56:33,295 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-03-05 19:56:33,295 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=50.56 KB heapSize=78.23 KB 2023-03-05 19:56:33,295 DEBUG [RS:0;jenkins-hbase4:37037] regionserver.HRegionServer(1505): Waiting on 014dc240b491a64066ab7b01d2b21258, 04217a89b4b7147d2fd1e001ac928c17, 1588230740, 237c9aa4b41fcfa45032528ec18afe9b, 25af79a83c9b248c956b57fc8507ead6, 25c8546691095e1f6ab386a199ccadff, 25ed4806e34b655e8718967079efb560, 27d678a97a95df40c894e3fe4540a1f2, 2d86bbeed3970cf2266d199ff776cf8d, 319680d28f8da52640b97eff4eaa9239, 484f8ac3504f985fdafb904759fb56ee, 5f33741eb2f558bc4b10a65a4789a550, 6fdfc14e8c700cdbd0a252655cbff4b4, 806fed504069a9293ba794165a04fd64, 942afd2d0a4ceef025fd5a607956d53d, aa3a9f808cf1db1e0667a9fff093c3ec, b9c878f961e0a32027f8960cfe26b4eb, bf39f342faf613faca7f43b94cc73bc4, daa57ebba7c140c319a2f6facc725e4a, e6a63acce9d53b978339ac99badba6fa, f5738c13f081c7a8286ce16de3ec8a21 2023-03-05 19:56:33,298 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/942afd2d0a4ceef025fd5a607956d53d/recovered.edits/38.seqid, newMaxSeqId=38, maxSeqId=31 2023-03-05 19:56:33,299 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,1,1678046116082.942afd2d0a4ceef025fd5a607956d53d. 2023-03-05 19:56:33,299 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 942afd2d0a4ceef025fd5a607956d53d: 2023-03-05 19:56:33,299 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testCompactionWithHFileLink,1,1678046116082.942afd2d0a4ceef025fd5a607956d53d. 2023-03-05 19:56:33,299 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing bf39f342faf613faca7f43b94cc73bc4, disabling compactions & flushes 2023-03-05 19:56:33,299 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678046078948.bf39f342faf613faca7f43b94cc73bc4. 2023-03-05 19:56:33,299 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678046078948.bf39f342faf613faca7f43b94cc73bc4. 2023-03-05 19:56:33,299 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678046078948.bf39f342faf613faca7f43b94cc73bc4. after waiting 0 ms 2023-03-05 19:56:33,299 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678046078948.bf39f342faf613faca7f43b94cc73bc4. 2023-03-05 19:56:33,304 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/bf39f342faf613faca7f43b94cc73bc4/recovered.edits/58.seqid, newMaxSeqId=58, maxSeqId=55 2023-03-05 19:56:33,305 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678046078948.bf39f342faf613faca7f43b94cc73bc4. 2023-03-05 19:56:33,305 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for bf39f342faf613faca7f43b94cc73bc4: 2023-03-05 19:56:33,306 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678046078948.bf39f342faf613faca7f43b94cc73bc4. 2023-03-05 19:56:33,306 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 25ed4806e34b655e8718967079efb560, disabling compactions & flushes 2023-03-05 19:56:33,306 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region ns:testMinorCompaction,2,1678046144232.25ed4806e34b655e8718967079efb560. 2023-03-05 19:56:33,306 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on ns:testMinorCompaction,2,1678046144232.25ed4806e34b655e8718967079efb560. 2023-03-05 19:56:33,306 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on ns:testMinorCompaction,2,1678046144232.25ed4806e34b655e8718967079efb560. after waiting 0 ms 2023-03-05 19:56:33,306 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region ns:testMinorCompaction,2,1678046144232.25ed4806e34b655e8718967079efb560. 2023-03-05 19:56:33,311 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=48.28 KB at sequenceid=130 (bloomFilter=false), to=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/hbase/meta/1588230740/.tmp/info/f8a747a081454c2db778148d88c4487f 2023-03-05 19:56:33,314 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f8a747a081454c2db778148d88c4487f 2023-03-05 19:56:33,316 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/25ed4806e34b655e8718967079efb560/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-03-05 19:56:33,317 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed ns:testMinorCompaction,2,1678046144232.25ed4806e34b655e8718967079efb560. 2023-03-05 19:56:33,317 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 25ed4806e34b655e8718967079efb560: 2023-03-05 19:56:33,317 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed ns:testMinorCompaction,2,1678046144232.25ed4806e34b655e8718967079efb560. 2023-03-05 19:56:33,318 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing f5738c13f081c7a8286ce16de3ec8a21, disabling compactions & flushes 2023-03-05 19:56:33,318 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region ns:testMinorCompaction,1,1678046144232.f5738c13f081c7a8286ce16de3ec8a21. 2023-03-05 19:56:33,318 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on ns:testMinorCompaction,1,1678046144232.f5738c13f081c7a8286ce16de3ec8a21. 2023-03-05 19:56:33,318 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on ns:testMinorCompaction,1,1678046144232.f5738c13f081c7a8286ce16de3ec8a21. after waiting 0 ms 2023-03-05 19:56:33,318 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region ns:testMinorCompaction,1,1678046144232.f5738c13f081c7a8286ce16de3ec8a21. 2023-03-05 19:56:33,322 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/f5738c13f081c7a8286ce16de3ec8a21/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-03-05 19:56:33,323 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed ns:testMinorCompaction,1,1678046144232.f5738c13f081c7a8286ce16de3ec8a21. 2023-03-05 19:56:33,323 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for f5738c13f081c7a8286ce16de3ec8a21: 2023-03-05 19:56:33,323 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed ns:testMinorCompaction,1,1678046144232.f5738c13f081c7a8286ce16de3ec8a21. 2023-03-05 19:56:33,323 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 6fdfc14e8c700cdbd0a252655cbff4b4, disabling compactions & flushes 2023-03-05 19:56:33,323 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region ns:testMinorCompaction,,1678046144232.6fdfc14e8c700cdbd0a252655cbff4b4. 2023-03-05 19:56:33,323 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on ns:testMinorCompaction,,1678046144232.6fdfc14e8c700cdbd0a252655cbff4b4. 2023-03-05 19:56:33,323 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on ns:testMinorCompaction,,1678046144232.6fdfc14e8c700cdbd0a252655cbff4b4. after waiting 0 ms 2023-03-05 19:56:33,323 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region ns:testMinorCompaction,,1678046144232.6fdfc14e8c700cdbd0a252655cbff4b4. 2023-03-05 19:56:33,326 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/ns/testMinorCompaction/6fdfc14e8c700cdbd0a252655cbff4b4/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-03-05 19:56:33,327 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed ns:testMinorCompaction,,1678046144232.6fdfc14e8c700cdbd0a252655cbff4b4. 2023-03-05 19:56:33,327 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 6fdfc14e8c700cdbd0a252655cbff4b4: 2023-03-05 19:56:33,327 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed ns:testMinorCompaction,,1678046144232.6fdfc14e8c700cdbd0a252655cbff4b4. 2023-03-05 19:56:33,327 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 484f8ac3504f985fdafb904759fb56ee, disabling compactions & flushes 2023-03-05 19:56:33,327 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionFromAdmin,,1678046174038.484f8ac3504f985fdafb904759fb56ee. 2023-03-05 19:56:33,327 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionFromAdmin,,1678046174038.484f8ac3504f985fdafb904759fb56ee. 2023-03-05 19:56:33,327 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionFromAdmin,,1678046174038.484f8ac3504f985fdafb904759fb56ee. after waiting 0 ms 2023-03-05 19:56:33,327 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionFromAdmin,,1678046174038.484f8ac3504f985fdafb904759fb56ee. 2023-03-05 19:56:33,330 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/484f8ac3504f985fdafb904759fb56ee/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-03-05 19:56:33,331 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionFromAdmin,,1678046174038.484f8ac3504f985fdafb904759fb56ee. 2023-03-05 19:56:33,331 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 484f8ac3504f985fdafb904759fb56ee: 2023-03-05 19:56:33,331 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionFromAdmin,,1678046174038.484f8ac3504f985fdafb904759fb56ee. 2023-03-05 19:56:33,331 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 04217a89b4b7147d2fd1e001ac928c17, disabling compactions & flushes 2023-03-05 19:56:33,331 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthly,,1678046023317.04217a89b4b7147d2fd1e001ac928c17. 2023-03-05 19:56:33,331 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1678046023317.04217a89b4b7147d2fd1e001ac928c17. 2023-03-05 19:56:33,331 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1678046023317.04217a89b4b7147d2fd1e001ac928c17. after waiting 0 ms 2023-03-05 19:56:33,331 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthly,,1678046023317.04217a89b4b7147d2fd1e001ac928c17. 2023-03-05 19:56:33,334 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/04217a89b4b7147d2fd1e001ac928c17/recovered.edits/55.seqid, newMaxSeqId=55, maxSeqId=49 2023-03-05 19:56:33,335 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthly,,1678046023317.04217a89b4b7147d2fd1e001ac928c17. 2023-03-05 19:56:33,335 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 04217a89b4b7147d2fd1e001ac928c17: 2023-03-05 19:56:33,335 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithWeeklyFollowedByMonthly,,1678046023317.04217a89b4b7147d2fd1e001ac928c17. 2023-03-05 19:56:33,335 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 319680d28f8da52640b97eff4eaa9239, disabling compactions & flushes 2023-03-05 19:56:33,335 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testScannerOnBulkLoadRefHFiles,1,1678046061330.319680d28f8da52640b97eff4eaa9239. 2023-03-05 19:56:33,335 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,1,1678046061330.319680d28f8da52640b97eff4eaa9239. 2023-03-05 19:56:33,335 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testScannerOnBulkLoadRefHFiles,1,1678046061330.319680d28f8da52640b97eff4eaa9239. after waiting 0 ms 2023-03-05 19:56:33,335 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testScannerOnBulkLoadRefHFiles,1,1678046061330.319680d28f8da52640b97eff4eaa9239. 2023-03-05 19:56:33,337 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testScannerOnBulkLoadRefHFiles/319680d28f8da52640b97eff4eaa9239/recovered.edits/6.seqid, newMaxSeqId=6, maxSeqId=1 2023-03-05 19:56:33,338 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testScannerOnBulkLoadRefHFiles,1,1678046061330.319680d28f8da52640b97eff4eaa9239. 2023-03-05 19:56:33,338 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 319680d28f8da52640b97eff4eaa9239: 2023-03-05 19:56:33,338 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testScannerOnBulkLoadRefHFiles,1,1678046061330.319680d28f8da52640b97eff4eaa9239. 2023-03-05 19:56:33,338 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing aa3a9f808cf1db1e0667a9fff093c3ec, disabling compactions & flushes 2023-03-05 19:56:33,338 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithMonthlyPolicy,,1678046035983.aa3a9f808cf1db1e0667a9fff093c3ec. 2023-03-05 19:56:33,338 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithMonthlyPolicy,,1678046035983.aa3a9f808cf1db1e0667a9fff093c3ec. 2023-03-05 19:56:33,338 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithMonthlyPolicy,,1678046035983.aa3a9f808cf1db1e0667a9fff093c3ec. after waiting 0 ms 2023-03-05 19:56:33,339 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithMonthlyPolicy,,1678046035983.aa3a9f808cf1db1e0667a9fff093c3ec. 2023-03-05 19:56:33,339 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-03-05 19:56:33,341 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithMonthlyPolicy/aa3a9f808cf1db1e0667a9fff093c3ec/recovered.edits/52.seqid, newMaxSeqId=52, maxSeqId=1 2023-03-05 19:56:33,342 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithMonthlyPolicy,,1678046035983.aa3a9f808cf1db1e0667a9fff093c3ec. 2023-03-05 19:56:33,342 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for aa3a9f808cf1db1e0667a9fff093c3ec: 2023-03-05 19:56:33,342 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithMonthlyPolicy,,1678046035983.aa3a9f808cf1db1e0667a9fff093c3ec. 2023-03-05 19:56:33,342 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 25af79a83c9b248c956b57fc8507ead6, disabling compactions & flushes 2023-03-05 19:56:33,342 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,2,1678046116082.25af79a83c9b248c956b57fc8507ead6. 2023-03-05 19:56:33,342 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,2,1678046116082.25af79a83c9b248c956b57fc8507ead6. 2023-03-05 19:56:33,342 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,2,1678046116082.25af79a83c9b248c956b57fc8507ead6. after waiting 0 ms 2023-03-05 19:56:33,342 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,2,1678046116082.25af79a83c9b248c956b57fc8507ead6. 2023-03-05 19:56:33,345 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/25af79a83c9b248c956b57fc8507ead6/recovered.edits/38.seqid, newMaxSeqId=38, maxSeqId=31 2023-03-05 19:56:33,346 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,2,1678046116082.25af79a83c9b248c956b57fc8507ead6. 2023-03-05 19:56:33,346 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 25af79a83c9b248c956b57fc8507ead6: 2023-03-05 19:56:33,346 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testCompactionWithHFileLink,2,1678046116082.25af79a83c9b248c956b57fc8507ead6. 2023-03-05 19:56:33,346 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 014dc240b491a64066ab7b01d2b21258, disabling compactions & flushes 2023-03-05 19:56:33,346 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMinorCompactionWithMonthlyPolicy,,1678046161366.014dc240b491a64066ab7b01d2b21258. 2023-03-05 19:56:33,346 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMinorCompactionWithMonthlyPolicy,,1678046161366.014dc240b491a64066ab7b01d2b21258. 2023-03-05 19:56:33,346 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMinorCompactionWithMonthlyPolicy,,1678046161366.014dc240b491a64066ab7b01d2b21258. after waiting 0 ms 2023-03-05 19:56:33,346 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMinorCompactionWithMonthlyPolicy,,1678046161366.014dc240b491a64066ab7b01d2b21258. 2023-03-05 19:56:33,348 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithMonthlyPolicy/014dc240b491a64066ab7b01d2b21258/recovered.edits/52.seqid, newMaxSeqId=52, maxSeqId=1 2023-03-05 19:56:33,349 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMinorCompactionWithMonthlyPolicy,,1678046161366.014dc240b491a64066ab7b01d2b21258. 2023-03-05 19:56:33,349 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 014dc240b491a64066ab7b01d2b21258: 2023-03-05 19:56:33,349 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMinorCompactionWithMonthlyPolicy,,1678046161366.014dc240b491a64066ab7b01d2b21258. 2023-03-05 19:56:33,349 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 27d678a97a95df40c894e3fe4540a1f2, disabling compactions & flushes 2023-03-05 19:56:33,349 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionFromAdmin,2,1678046174038.27d678a97a95df40c894e3fe4540a1f2. 2023-03-05 19:56:33,349 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionFromAdmin,2,1678046174038.27d678a97a95df40c894e3fe4540a1f2. 2023-03-05 19:56:33,349 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionFromAdmin,2,1678046174038.27d678a97a95df40c894e3fe4540a1f2. after waiting 0 ms 2023-03-05 19:56:33,349 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionFromAdmin,2,1678046174038.27d678a97a95df40c894e3fe4540a1f2. 2023-03-05 19:56:33,352 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/27d678a97a95df40c894e3fe4540a1f2/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-03-05 19:56:33,353 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionFromAdmin,2,1678046174038.27d678a97a95df40c894e3fe4540a1f2. 2023-03-05 19:56:33,353 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 27d678a97a95df40c894e3fe4540a1f2: 2023-03-05 19:56:33,353 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionFromAdmin,2,1678046174038.27d678a97a95df40c894e3fe4540a1f2. 2023-03-05 19:56:33,353 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing e6a63acce9d53b978339ac99badba6fa, disabling compactions & flushes 2023-03-05 19:56:33,353 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMinorCompactionWithWeeklyPolicy,,1678046048748.e6a63acce9d53b978339ac99badba6fa. 2023-03-05 19:56:33,353 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMinorCompactionWithWeeklyPolicy,,1678046048748.e6a63acce9d53b978339ac99badba6fa. 2023-03-05 19:56:33,353 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMinorCompactionWithWeeklyPolicy,,1678046048748.e6a63acce9d53b978339ac99badba6fa. after waiting 0 ms 2023-03-05 19:56:33,353 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMinorCompactionWithWeeklyPolicy,,1678046048748.e6a63acce9d53b978339ac99badba6fa. 2023-03-05 19:56:33,356 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMinorCompactionWithWeeklyPolicy/e6a63acce9d53b978339ac99badba6fa/recovered.edits/49.seqid, newMaxSeqId=49, maxSeqId=1 2023-03-05 19:56:33,357 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMinorCompactionWithWeeklyPolicy,,1678046048748.e6a63acce9d53b978339ac99badba6fa. 2023-03-05 19:56:33,357 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for e6a63acce9d53b978339ac99badba6fa: 2023-03-05 19:56:33,357 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMinorCompactionWithWeeklyPolicy,,1678046048748.e6a63acce9d53b978339ac99badba6fa. 2023-03-05 19:56:33,357 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 5f33741eb2f558bc4b10a65a4789a550, disabling compactions & flushes 2023-03-05 19:56:33,357 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyPolicy,,1678046065984.5f33741eb2f558bc4b10a65a4789a550. 2023-03-05 19:56:33,357 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyPolicy,,1678046065984.5f33741eb2f558bc4b10a65a4789a550. 2023-03-05 19:56:33,357 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyPolicy,,1678046065984.5f33741eb2f558bc4b10a65a4789a550. after waiting 0 ms 2023-03-05 19:56:33,357 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyPolicy,,1678046065984.5f33741eb2f558bc4b10a65a4789a550. 2023-03-05 19:56:33,359 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyPolicy/5f33741eb2f558bc4b10a65a4789a550/recovered.edits/50.seqid, newMaxSeqId=50, maxSeqId=1 2023-03-05 19:56:33,360 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyPolicy,,1678046065984.5f33741eb2f558bc4b10a65a4789a550. 2023-03-05 19:56:33,360 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 5f33741eb2f558bc4b10a65a4789a550: 2023-03-05 19:56:33,360 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithWeeklyPolicy,,1678046065984.5f33741eb2f558bc4b10a65a4789a550. 2023-03-05 19:56:33,360 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 237c9aa4b41fcfa45032528ec18afe9b, disabling compactions & flushes 2023-03-05 19:56:33,360 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testScannerOnBulkLoadRefHFiles,,1678046061330.237c9aa4b41fcfa45032528ec18afe9b. 2023-03-05 19:56:33,360 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,,1678046061330.237c9aa4b41fcfa45032528ec18afe9b. 2023-03-05 19:56:33,360 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testScannerOnBulkLoadRefHFiles,,1678046061330.237c9aa4b41fcfa45032528ec18afe9b. after waiting 0 ms 2023-03-05 19:56:33,360 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testScannerOnBulkLoadRefHFiles,,1678046061330.237c9aa4b41fcfa45032528ec18afe9b. 2023-03-05 19:56:33,362 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testScannerOnBulkLoadRefHFiles/237c9aa4b41fcfa45032528ec18afe9b/recovered.edits/6.seqid, newMaxSeqId=6, maxSeqId=1 2023-03-05 19:56:33,362 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testScannerOnBulkLoadRefHFiles,,1678046061330.237c9aa4b41fcfa45032528ec18afe9b. 2023-03-05 19:56:33,362 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 237c9aa4b41fcfa45032528ec18afe9b: 2023-03-05 19:56:33,362 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testScannerOnBulkLoadRefHFiles,,1678046061330.237c9aa4b41fcfa45032528ec18afe9b. 2023-03-05 19:56:33,362 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing b9c878f961e0a32027f8960cfe26b4eb, disabling compactions & flushes 2023-03-05 19:56:33,362 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678046094042.b9c878f961e0a32027f8960cfe26b4eb. 2023-03-05 19:56:33,362 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678046094042.b9c878f961e0a32027f8960cfe26b4eb. 2023-03-05 19:56:33,362 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678046094042.b9c878f961e0a32027f8960cfe26b4eb. after waiting 0 ms 2023-03-05 19:56:33,362 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678046094042.b9c878f961e0a32027f8960cfe26b4eb. 2023-03-05 19:56:33,364 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/b9c878f961e0a32027f8960cfe26b4eb/recovered.edits/58.seqid, newMaxSeqId=58, maxSeqId=55 2023-03-05 19:56:33,365 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678046094042.b9c878f961e0a32027f8960cfe26b4eb. 2023-03-05 19:56:33,365 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for b9c878f961e0a32027f8960cfe26b4eb: 2023-03-05 19:56:33,365 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678046094042.b9c878f961e0a32027f8960cfe26b4eb. 2023-03-05 19:56:33,365 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 25c8546691095e1f6ab386a199ccadff, disabling compactions & flushes 2023-03-05 19:56:33,365 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1678046022498.25c8546691095e1f6ab386a199ccadff. 2023-03-05 19:56:33,365 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1678046022498.25c8546691095e1f6ab386a199ccadff. 2023-03-05 19:56:33,365 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1678046022498.25c8546691095e1f6ab386a199ccadff. after waiting 0 ms 2023-03-05 19:56:33,365 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1678046022498.25c8546691095e1f6ab386a199ccadff. 2023-03-05 19:56:33,365 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 25c8546691095e1f6ab386a199ccadff 1/1 column families, dataSize=109 B heapSize=592 B 2023-03-05 19:56:33,386 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactedHFilesCleaner was stopped 2023-03-05 19:56:33,495 DEBUG [RS:0;jenkins-hbase4:37037] regionserver.HRegionServer(1505): Waiting on 1588230740, 25c8546691095e1f6ab386a199ccadff, 2d86bbeed3970cf2266d199ff776cf8d, 806fed504069a9293ba794165a04fd64, daa57ebba7c140c319a2f6facc725e4a 2023-03-05 19:56:33,696 DEBUG [RS:0;jenkins-hbase4:37037] regionserver.HRegionServer(1505): Waiting on 1588230740, 25c8546691095e1f6ab386a199ccadff, 2d86bbeed3970cf2266d199ff776cf8d, 806fed504069a9293ba794165a04fd64, daa57ebba7c140c319a2f6facc725e4a 2023-03-05 19:56:33,721 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=102 B at sequenceid=130 (bloomFilter=false), to=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/hbase/meta/1588230740/.tmp/rep_barrier/90ec653954124f609c813a72460db730 2023-03-05 19:56:33,727 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 90ec653954124f609c813a72460db730 2023-03-05 19:56:33,770 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=109 B at sequenceid=7 (bloomFilter=true), to=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/hbase/namespace/25c8546691095e1f6ab386a199ccadff/.tmp/info/5631585fcced4a22971ca29241d69406 2023-03-05 19:56:33,775 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/hbase/namespace/25c8546691095e1f6ab386a199ccadff/.tmp/info/5631585fcced4a22971ca29241d69406 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/hbase/namespace/25c8546691095e1f6ab386a199ccadff/info/5631585fcced4a22971ca29241d69406 2023-03-05 19:56:33,778 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/hbase/namespace/25c8546691095e1f6ab386a199ccadff/info/5631585fcced4a22971ca29241d69406, entries=3, sequenceid=7, filesize=4.8 K 2023-03-05 19:56:33,779 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~109 B/109, heapSize ~576 B/576, currentSize=0 B/0 for 25c8546691095e1f6ab386a199ccadff in 414ms, sequenceid=7, compaction requested=false 2023-03-05 19:56:33,782 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/hbase/namespace/25c8546691095e1f6ab386a199ccadff/recovered.edits/10.seqid, newMaxSeqId=10, maxSeqId=1 2023-03-05 19:56:33,783 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1678046022498.25c8546691095e1f6ab386a199ccadff. 2023-03-05 19:56:33,783 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 25c8546691095e1f6ab386a199ccadff: 2023-03-05 19:56:33,783 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1678046022498.25c8546691095e1f6ab386a199ccadff. 2023-03-05 19:56:33,783 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 2d86bbeed3970cf2266d199ff776cf8d, disabling compactions & flushes 2023-03-05 19:56:33,783 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testScannerOnBulkLoadRefHFiles,2,1678046061330.2d86bbeed3970cf2266d199ff776cf8d. 2023-03-05 19:56:33,783 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,2,1678046061330.2d86bbeed3970cf2266d199ff776cf8d. 2023-03-05 19:56:33,783 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testScannerOnBulkLoadRefHFiles,2,1678046061330.2d86bbeed3970cf2266d199ff776cf8d. after waiting 0 ms 2023-03-05 19:56:33,783 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testScannerOnBulkLoadRefHFiles,2,1678046061330.2d86bbeed3970cf2266d199ff776cf8d. 2023-03-05 19:56:33,786 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testScannerOnBulkLoadRefHFiles/2d86bbeed3970cf2266d199ff776cf8d/recovered.edits/15.seqid, newMaxSeqId=15, maxSeqId=1 2023-03-05 19:56:33,786 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testScannerOnBulkLoadRefHFiles,2,1678046061330.2d86bbeed3970cf2266d199ff776cf8d. 2023-03-05 19:56:33,786 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 2d86bbeed3970cf2266d199ff776cf8d: 2023-03-05 19:56:33,786 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testScannerOnBulkLoadRefHFiles,2,1678046061330.2d86bbeed3970cf2266d199ff776cf8d. 2023-03-05 19:56:33,786 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 806fed504069a9293ba794165a04fd64, disabling compactions & flushes 2023-03-05 19:56:33,786 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionFromAdmin,1,1678046174038.806fed504069a9293ba794165a04fd64. 2023-03-05 19:56:33,786 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionFromAdmin,1,1678046174038.806fed504069a9293ba794165a04fd64. 2023-03-05 19:56:33,786 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionFromAdmin,1,1678046174038.806fed504069a9293ba794165a04fd64. after waiting 0 ms 2023-03-05 19:56:33,786 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionFromAdmin,1,1678046174038.806fed504069a9293ba794165a04fd64. 2023-03-05 19:56:33,789 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testMajorCompactionFromAdmin/806fed504069a9293ba794165a04fd64/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-03-05 19:56:33,789 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionFromAdmin,1,1678046174038.806fed504069a9293ba794165a04fd64. 2023-03-05 19:56:33,789 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 806fed504069a9293ba794165a04fd64: 2023-03-05 19:56:33,789 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionFromAdmin,1,1678046174038.806fed504069a9293ba794165a04fd64. 2023-03-05 19:56:33,789 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing daa57ebba7c140c319a2f6facc725e4a, disabling compactions & flushes 2023-03-05 19:56:33,789 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,,1678046116082.daa57ebba7c140c319a2f6facc725e4a. 2023-03-05 19:56:33,789 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,,1678046116082.daa57ebba7c140c319a2f6facc725e4a. 2023-03-05 19:56:33,789 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,,1678046116082.daa57ebba7c140c319a2f6facc725e4a. after waiting 0 ms 2023-03-05 19:56:33,789 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,,1678046116082.daa57ebba7c140c319a2f6facc725e4a. 2023-03-05 19:56:33,792 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/default/testCompactionWithHFileLink/daa57ebba7c140c319a2f6facc725e4a/recovered.edits/38.seqid, newMaxSeqId=38, maxSeqId=31 2023-03-05 19:56:33,792 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,,1678046116082.daa57ebba7c140c319a2f6facc725e4a. 2023-03-05 19:56:33,792 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for daa57ebba7c140c319a2f6facc725e4a: 2023-03-05 19:56:33,792 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testCompactionWithHFileLink,,1678046116082.daa57ebba7c140c319a2f6facc725e4a. 2023-03-05 19:56:33,838 DEBUG [regionserver/jenkins-hbase4:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-03-05 19:56:33,896 DEBUG [RS:0;jenkins-hbase4:37037] regionserver.HRegionServer(1505): Waiting on 1588230740 2023-03-05 19:56:34,096 DEBUG [RS:0;jenkins-hbase4:37037] regionserver.HRegionServer(1505): Waiting on 1588230740 2023-03-05 19:56:34,134 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.18 KB at sequenceid=130 (bloomFilter=false), to=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/hbase/meta/1588230740/.tmp/table/5e83ea8a3d9a48998eb55d2258f231eb 2023-03-05 19:56:34,137 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5e83ea8a3d9a48998eb55d2258f231eb 2023-03-05 19:56:34,137 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/hbase/meta/1588230740/.tmp/info/f8a747a081454c2db778148d88c4487f as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/hbase/meta/1588230740/info/f8a747a081454c2db778148d88c4487f 2023-03-05 19:56:34,140 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f8a747a081454c2db778148d88c4487f 2023-03-05 19:56:34,140 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/hbase/meta/1588230740/info/f8a747a081454c2db778148d88c4487f, entries=226, sequenceid=130, filesize=37.9 K 2023-03-05 19:56:34,140 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/hbase/meta/1588230740/.tmp/rep_barrier/90ec653954124f609c813a72460db730 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/hbase/meta/1588230740/rep_barrier/90ec653954124f609c813a72460db730 2023-03-05 19:56:34,143 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 90ec653954124f609c813a72460db730 2023-03-05 19:56:34,143 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/hbase/meta/1588230740/rep_barrier/90ec653954124f609c813a72460db730, entries=1, sequenceid=130, filesize=5.0 K 2023-03-05 19:56:34,144 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/hbase/meta/1588230740/.tmp/table/5e83ea8a3d9a48998eb55d2258f231eb as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/hbase/meta/1588230740/table/5e83ea8a3d9a48998eb55d2258f231eb 2023-03-05 19:56:34,147 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5e83ea8a3d9a48998eb55d2258f231eb 2023-03-05 19:56:34,148 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/hbase/meta/1588230740/table/5e83ea8a3d9a48998eb55d2258f231eb, entries=27, sequenceid=130, filesize=6.6 K 2023-03-05 19:56:34,150 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~50.56 KB/51772, heapSize ~78.19 KB/80064, currentSize=0 B/0 for 1588230740 in 855ms, sequenceid=130, compaction requested=false 2023-03-05 19:56:34,154 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/data/hbase/meta/1588230740/recovered.edits/133.seqid, newMaxSeqId=133, maxSeqId=1 2023-03-05 19:56:34,154 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-03-05 19:56:34,154 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-03-05 19:56:34,154 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-03-05 19:56:34,154 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-03-05 19:56:34,200 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactionChecker was stopped 2023-03-05 19:56:34,200 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: MemstoreFlusherChore was stopped 2023-03-05 19:56:34,297 INFO [RS:0;jenkins-hbase4:37037] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,37037,1678046018983; all regions closed. 2023-03-05 19:56:34,304 DEBUG [RS:0;jenkins-hbase4:37037] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/oldWALs 2023-03-05 19:56:34,304 INFO [RS:0;jenkins-hbase4:37037] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C37037%2C1678046018983.meta:.meta(num 1678046022254) 2023-03-05 19:56:34,307 DEBUG [RS:0;jenkins-hbase4:37037] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/oldWALs 2023-03-05 19:56:34,307 INFO [RS:0;jenkins-hbase4:37037] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C37037%2C1678046018983:(num 1678046021355) 2023-03-05 19:56:34,307 DEBUG [RS:0;jenkins-hbase4:37037] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-05 19:56:34,307 INFO [RS:0;jenkins-hbase4:37037] regionserver.LeaseManager(133): Closed leases 2023-03-05 19:56:34,308 INFO [RS:0;jenkins-hbase4:37037] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-03-05 19:56:34,308 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-03-05 19:56:34,309 INFO [RS:0;jenkins-hbase4:37037] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:37037 2023-03-05 19:56:34,318 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-03-05 19:56:34,318 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,37037,1678046018983 2023-03-05 19:56:34,318 ERROR [Listener at localhost/40787-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@f5a9d99 rejected from java.util.concurrent.ThreadPoolExecutor@b6518aa[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 532] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-03-05 19:56:34,319 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-03-05 19:56:34,319 ERROR [Listener at localhost/40787-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@240e3e17 rejected from java.util.concurrent.ThreadPoolExecutor@b6518aa[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 532] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-03-05 19:56:34,320 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,37037,1678046018983] 2023-03-05 19:56:34,320 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,37037,1678046018983; numProcessing=1 2023-03-05 19:56:34,321 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,37037,1678046018983 already deleted, retry=false 2023-03-05 19:56:34,322 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,37037,1678046018983 expired; onlineServers=0 2023-03-05 19:56:34,322 INFO [RegionServerTracker-0] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,38221,1678046017738' ***** 2023-03-05 19:56:34,322 INFO [RegionServerTracker-0] regionserver.HRegionServer(2310): STOPPED: Cluster shutdown set; onlineServer=0 2023-03-05 19:56:34,322 DEBUG [M:0;jenkins-hbase4:38221] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@73a4bdd0, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-03-05 19:56:34,322 INFO [M:0;jenkins-hbase4:38221] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,38221,1678046017738 2023-03-05 19:56:34,322 INFO [M:0;jenkins-hbase4:38221] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,38221,1678046017738; all regions closed. 2023-03-05 19:56:34,322 DEBUG [M:0;jenkins-hbase4:38221] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-05 19:56:34,322 DEBUG [M:0;jenkins-hbase4:38221] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-03-05 19:56:34,322 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-03-05 19:56:34,322 DEBUG [M:0;jenkins-hbase4:38221] cleaner.HFileCleaner(317): Stopping file delete threads 2023-03-05 19:56:34,323 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1678046020989] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1678046020989,5,FailOnTimeoutGroup] 2023-03-05 19:56:34,323 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1678046020990] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1678046020990,5,FailOnTimeoutGroup] 2023-03-05 19:56:34,323 INFO [M:0;jenkins-hbase4:38221] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-03-05 19:56:34,323 INFO [M:0;jenkins-hbase4:38221] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-03-05 19:56:34,323 INFO [M:0;jenkins-hbase4:38221] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-03-05 19:56:34,323 DEBUG [M:0;jenkins-hbase4:38221] master.HMaster(1512): Stopping service threads 2023-03-05 19:56:34,324 INFO [M:0;jenkins-hbase4:38221] procedure2.RemoteProcedureDispatcher(118): Stopping procedure remote dispatcher 2023-03-05 19:56:34,324 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-03-05 19:56:34,324 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-05 19:56:34,324 ERROR [M:0;jenkins-hbase4:38221] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-20,5,PEWorkerGroup] Thread[HFileArchiver-21,5,PEWorkerGroup] Thread[HFileArchiver-22,5,PEWorkerGroup] 2023-03-05 19:56:34,324 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-03-05 19:56:34,325 INFO [M:0;jenkins-hbase4:38221] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-03-05 19:56:34,325 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-03-05 19:56:34,325 DEBUG [M:0;jenkins-hbase4:38221] zookeeper.ZKUtil(398): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Unable to get data of znode /hbase/master because node does not exist (not an error) 2023-03-05 19:56:34,325 WARN [M:0;jenkins-hbase4:38221] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-03-05 19:56:34,325 INFO [M:0;jenkins-hbase4:38221] assignment.AssignmentManager(315): Stopping assignment manager 2023-03-05 19:56:34,325 INFO [M:0;jenkins-hbase4:38221] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-03-05 19:56:34,325 DEBUG [M:0;jenkins-hbase4:38221] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-03-05 19:56:34,325 INFO [M:0;jenkins-hbase4:38221] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-03-05 19:56:34,325 DEBUG [M:0;jenkins-hbase4:38221] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-03-05 19:56:34,325 DEBUG [M:0;jenkins-hbase4:38221] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-03-05 19:56:34,325 DEBUG [M:0;jenkins-hbase4:38221] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-03-05 19:56:34,325 INFO [M:0;jenkins-hbase4:38221] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=512.73 KB heapSize=598.37 KB 2023-03-05 19:56:34,332 INFO [M:0;jenkins-hbase4:38221] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=512.73 KB at sequenceid=1005 (bloomFilter=true), to=hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/76825c9293664421add842668b2469c0 2023-03-05 19:56:34,335 INFO [M:0;jenkins-hbase4:38221] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 76825c9293664421add842668b2469c0 2023-03-05 19:56:34,336 DEBUG [M:0;jenkins-hbase4:38221] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/76825c9293664421add842668b2469c0 as hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/76825c9293664421add842668b2469c0 2023-03-05 19:56:34,339 INFO [M:0;jenkins-hbase4:38221] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 76825c9293664421add842668b2469c0 2023-03-05 19:56:34,339 INFO [M:0;jenkins-hbase4:38221] regionserver.HStore(1080): Added hdfs://localhost:32809/user/jenkins/test-data/99c5431f-31d4-5724-d414-db244f142743/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/76825c9293664421add842668b2469c0, entries=123, sequenceid=1005, filesize=33.3 K 2023-03-05 19:56:34,339 INFO [M:0;jenkins-hbase4:38221] regionserver.HRegion(2948): Finished flush of dataSize ~512.73 KB/525040, heapSize ~598.35 KB/612712, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 14ms, sequenceid=1005, compaction requested=false 2023-03-05 19:56:34,340 INFO [M:0;jenkins-hbase4:38221] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-03-05 19:56:34,340 DEBUG [M:0;jenkins-hbase4:38221] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-03-05 19:56:34,343 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-03-05 19:56:34,343 INFO [M:0;jenkins-hbase4:38221] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-03-05 19:56:34,343 INFO [M:0;jenkins-hbase4:38221] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:38221 2023-03-05 19:56:34,347 DEBUG [M:0;jenkins-hbase4:38221] zookeeper.RecoverableZooKeeper(172): Node /hbase/rs/jenkins-hbase4.apache.org,38221,1678046017738 already deleted, retry=false 2023-03-05 19:56:34,420 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-03-05 19:56:34,420 INFO [RS:0;jenkins-hbase4:37037] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,37037,1678046018983; zookeeper connection closed. 2023-03-05 19:56:34,420 ERROR [Listener at localhost/40787-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@5ce02e54 rejected from java.util.concurrent.ThreadPoolExecutor@b6518aa[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 532] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-03-05 19:56:34,420 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@19f75a32] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@19f75a32 2023-03-05 19:56:34,420 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): regionserver:37037-0x100b63e1f1d0001, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-03-05 19:56:34,421 ERROR [Listener at localhost/40787-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@7be305b5 rejected from java.util.concurrent.ThreadPoolExecutor@b6518aa[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 532] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-03-05 19:56:34,421 INFO [Listener at localhost/40787] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 1 regionserver(s) complete 2023-03-05 19:56:34,520 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-03-05 19:56:34,520 INFO [M:0;jenkins-hbase4:38221] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,38221,1678046017738; zookeeper connection closed. 2023-03-05 19:56:34,520 ERROR [Listener at localhost/40787-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@6080ca5f rejected from java.util.concurrent.ThreadPoolExecutor@3b58de1b[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 907] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-03-05 19:56:34,521 DEBUG [Listener at localhost/40787-EventThread] zookeeper.ZKWatcher(600): master:38221-0x100b63e1f1d0000, quorum=127.0.0.1:62957, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-03-05 19:56:34,521 ERROR [Listener at localhost/40787-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@3b3d0010 rejected from java.util.concurrent.ThreadPoolExecutor@3b58de1b[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 907] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-03-05 19:56:34,522 WARN [Listener at localhost/40787] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-03-05 19:56:34,526 INFO [Listener at localhost/40787] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-03-05 19:56:34,634 WARN [BP-875493781-172.31.14.131-1678046014457 heartbeating to localhost/127.0.0.1:32809] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-03-05 19:56:34,635 WARN [BP-875493781-172.31.14.131-1678046014457 heartbeating to localhost/127.0.0.1:32809] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-875493781-172.31.14.131-1678046014457 (Datanode Uuid d35f9466-99e1-410c-8caf-7fef1a39b184) service to localhost/127.0.0.1:32809 2023-03-05 19:56:34,637 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5db36781-e654-c932-96c1-0485fe524a4f/cluster_01318ba7-3e0b-e9a9-4c5c-f222cae89dc3/dfs/data/data1/current/BP-875493781-172.31.14.131-1678046014457] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-03-05 19:56:34,638 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5db36781-e654-c932-96c1-0485fe524a4f/cluster_01318ba7-3e0b-e9a9-4c5c-f222cae89dc3/dfs/data/data2/current/BP-875493781-172.31.14.131-1678046014457] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-03-05 19:56:34,662 INFO [Listener at localhost/40787] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-03-05 19:56:34,792 INFO [Listener at localhost/40787] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-03-05 19:56:34,874 INFO [Listener at localhost/40787] hbase.HBaseTestingUtility(1293): Minicluster is down